Who do Voodoo Science

Who is Using Voodoo Science?

or,  Don’t over-analyze your water quality samples because you might not like the answer.

Someone get me my blood pressure meds.

What set me off today?  It was one more program specifying the EPA 160.2 (TSS) analysis method (now designated as SM-2540-D) and questioning the use of ASTM D-3977-97 (SSC).  This is just a symptom of the problems with many jurisdictional protocols in the USA.  The underlying problem is that scientists have typically not been in charge of water quality in this country.  People tend to hide behind the advisory
guidance of the Environmental Protection Agency (EPA) which is really very well founded if you take the time to read it.  The EPA guidance on water quality says that suspended solids are simply a surrogate for numerous other pollutants, and that if you reduce solids, you can assume that you are reducing other constituents.  While this is not science, it is a nice simple solution, and it takes a leap of faith to believe that it is actually true. Still, it is the law, so to speak.  If you have not read, “Management Measure 5” from the EPA, it would probably be a good idea for you to read the guidance for yourself.  It flatly states that particulates “include suspended, settleable, and bedload solids”, goes on to say that one study showed “that the greatest mass of contaminants in highway runoff is found on particles in the 425 to 850 micron (μm) range”.  While this seem abundantly clear, the guidance goes on to foster some confusion by stating that “Generally, individual particles found in a TSS sample are 62 μm (0.062 mm) or less in diameter and classified as either silts or clays.” If you are involved at any level in permitting water quality structures, you should actually read this management measure.  It can be found here:

(http://www.epa.gov/owow/nps/urbanmm/index.html)

The guidance also explains why the ASTM analysis method (SSC or ASTM D-3977-97) is the preferred method for determining the true concentration of solids in water samples with any mix of particle sizes (not EPA 160.2 TSS).

As a matter of fact, the EPA 160.2 TSS analysis method is not only inaccurate, it is not repeatable and should not be trusted.  It typically cannot meet the QA/QC standards for testing, because it gives answers all over the map for aliquots drawn from the same sample. Essentially TSS is a partial sample analysis, where a portion of the sample is drawn out with a pipette (or poured out), and then analyzed.  The result of the partial sample is deemed to be representative of the whole sample, and it is multiplied by some factor to adjust it back up, as if it represented the whole sample.  If one tenth of the sample was used for the analysis, the result is multiplied by 10 (and any error is also increased by 10).  Of course, larger particles tend to settle more quickly, so are not likely to be collected in the subsample, but it is interesting to note that TSS can over-estimate concentrations if the lab practitioner tries to adjust for settling effects by taking a sample low in the water column.  With the subsample, the lab worker has to accurately read the amount of water withdrawn also.  SSC on the other hand uses the entire sample and pours it through a filter to get all of the solids, and measures all of the liquid.   It is both accurate and repeatable.

To rely on TSS data challenges the intellect.  First, you have to believe that all the pollution lies in small particles. I once asked for people to share studies to confirm this, and one nice engineer brought forward a report that cited 5 such studies with fractional analysis.  Fractional analysis is a technique where particles are sorted by size, and then the concentrations of various pollutants are reported for each particle size range.  The very credible report even stated that pollutants tended to attach to smaller particles.  Trouble is, that is not what the data in the studies showed.  I plotted the data to get a good visual look at it.  The data in the studies indicated that the highest concentrations were, in general, in the 75 to 150 micron ranges.  One study did show a peak in the 2 to 6 micron range (probably from clay adsorption of metals), but that same study showed another peak in the 75 to 150 micron range.  I am still looking for studies that demonstrate the small particle effect in real stormwater sediments (not laboratory clay samples).  To be sure, it is very expensive to do fractional analysis, and so, such studies are rare.  If you have one, or know of one, please let me know about it.

You do not have to be an expert, however, to understand how man’s influence on the impervious surfaces we drive and walk upon tend to link pollutants to things larger than a speck on a gnat’s hair.  (The blogger introduces a rusty alkaline battery, a lump of asphalt, and a roadway enterline reflective paint chip into evidence.)

If you believe that small particles are the problem, and you want to rely on EPA 160.2 TSS, you have to believe that it is not necessary to get the right answer to concentrations also.  If you had a thermometer that you relied on, and it read 25 degrees one minute, and 45 the next, you would not believe it.  Well, you might believe it, but if you were depending on it to keep your pipes from freezing, you would not rely on it.  Somehow, people have deluded themselves into using an analysis method (EPA 160.2 TSS) that is neither accurate nor reliable.  Why would anybody do this?  Well, because it agrees with what they believe. You can believe it is 45 degrees outside as much as you want to, but if it is really 25 degrees outside, your car radiator and your pipes will freeze if not protected.  If your watersheds are at stake (and they are), we better start relying on good science and people who have actually proven what they can (and cannot) do.

My ETV testing is a case in point, but I could point out the testing of others just as well.  Every credible test of a structural BMP shows results for solids concentrations using SSC analysis are always higher than TSS analysis.  Regulators tend to want to believe the TSS numbers because…  well…, just because.  It does line up quite nicely with what “Mr. Manual” says, and how they “feel” about things.  Feeling concerned about protecting the environment is a good thing, but you should toss out feeling and rely on real science to actually accomplish environmental protection.

It should be noted that the inaccuracy of the TSS analysis method can cut both ways.  If the effluent concentration is under-reported by TSS analysis, a BMP could appear to be more efficient than it actually is.   There is no defense for using a bad method to analyze perfectly good samples. Let’s take my reported numbers from the ETV test, and look at the actual samples, which they dried, weighed, and split into sand / fines fractions. My SSC removal number was reported at 89% (It was 90% if you care to do the math), and my TSS at 21%.  When they took the sand and silt splits and did the “Sum of Loads” (SOL) calculations in “Table 5-7 Particle Size Distribution Results” of my report, they determined that we had 1,400 pounds of sand in the inlet and only 24 pounds in the outlet for a sand removal rate of 98%.  The table also showed 200 pounds of fines in the inlet and 133 pounds in the outlet, for a removal rate of 34%.  Ok, let’s assume for the sake of argument that the sand, flotables, organics, trash, debris (nothing but the sand was actually measured) are completely clean and carry no pollution at all including no nutrients.  (That should boggle your mind.)  We will take a zero for the sand, pretending that it weighed nothing and contained nothing. We still removed 34% of the silt and clay by weight, which is much different than the reported 21% for the sands and fines combined.  As an analysis method for real stormwater, TSS is simply inaccurate.

Lest you think I protest too much, you should know that we have an excellent study in Highlands, NC, that used TSS as the analysis method. The study evaluated nutrients, metals, fecal coliform and many other constituents.  Of course, we did not get 89% removal efficiency with TSS analysis instead of SSC.  In reality, we got a 96% removal rating in that study when TSS analysis was used.  This really sounds good, and it silences the critics who believe that our SSC data is somehow tainted, but I would trust the 89% SSC efficiency rating before I trusted the 96% from TSS simply because the TSS analysis is not reliable.

(The blogger climbs up on his soapbox now.) The same folks that believe in using inaccurate analysis methods promote automatic acceptance at an assumed removal rate for their BMPs of choice. They have a water quality manual that is the gospel, and it clearly states that their preferred BMPs remove 80% (or whatever number someone told them was good) of all pollutants.  “In the manual = Approved”; next case. They truly believe that all bad things move in tiny storms, and are attached to small particles.  Amen. They developed these firmly held beliefs at seminars presented by “Credible.org”, and have certificates of completion to prove it. Selah.

Armed with this “data” they confront people who have actually done verification work at a certified laboratory, or been evaluated in a strictly regimented field testing program and label those people “unclean, for they resideth not in the hallowed manual.”  No matter that their manual-sanctioned practices have not been able to achieve the magic number when actually field tested (or have not been tested at all), or that their accepted methods have been proven inaccurate and outdated.  No, thou art in our house now, and thou shalt do as we say. (Climbing down off of the soapbox.)

I guess I would be ok with this if I did not get to look at the material that comes out of the devices we clean.  It is not clean, washed builder’s sand.  It is nasty.  It is full of pollution.  It also has more than 21% fines in it, but that is another story.  You can believe in voodoo science if you want to, but I would challenge you to go with good sampling, accurate analysis, and to get about the task of protecting our waterways.  It is ironic that my industry is accused of proffering “black box”, unproven solutions, when in reality; it is “Mr. Manual” and his crowd who are offering unproven “snake oil” remedies.