A small trial in your garden proves very little!

Don't you believe it.

 

My grandson Arthur Brook is a budding researcher

It is good to try out new ideas and how we do things in the garden. Without trial and error we would be stuck in a rut and never change.

What worries me is that sometimes gardeners construct 'little trials' and proclaim to the world they have found something new. Worse when they read dubious information that has no scientific validity - often promoted by advertising or someone with an axe to grind - they not only believe it but pass it on as a fact.

And where does that leave me who sets out to advise gardeners?

As a species we would have never have advanced without having been taught by others how to do things and to believe what we are told! It's this same inborn trust that leaves us very vulnerable to getting things wrong.

Although I am suggesting today that what you do in your garden should be scientifically valid, how do you know? There are many dubious 'scientific' facts and explanations peddled in horticulture. Even research as high as University level is sometimes flawed. All the more so when 'the publicity department' has got their hands on research information. Worse still, when research is interpreted by journalists who add their unproven world view as a thin veneer on what has been actually shown.

Nature provides its own trials – and tribulations

I have been involved with students who have done so called trials as part of their education. It is a good thing to have an investigative mind. I have never known them to actually prove anything!

There are too many variables. What seems to be a genuine result is often complete chance. To have scientific validity results need to be duplicated and statistically valid. Not only do experiments need a large number of repetitions in their make up, the actual experiments need to be repeatable by other investigators. How else does one avoid wishful thinking, bias, and very human error?

As an example suppose a gardener is testing a new product, what can go wrong?

If the actual product is used by millions of gardeners and perhaps used by a whole industry - perhaps a high nitrogen fertiliser - you will get a result in as much as the plant may grow quicker and turn a bright green! (But that won't tell you whether it is a good thing or a bad one).

On the other hand if that product is something new and the effects are small the very best you can show is it might work for you.

So many things can go wrong in a trial and the possible errors are legion!

Is there a control? I mean is there really a comparison where two test treatments are exactly the same other than the one variable being tested?

I mean exactly the same! Is the soil for the duplicated treatments identical? Is the drainage uniform? Are light levels the same? In a small garden illumination is very variable. What about the test plants - did they really start out exactly the same? They were of course the same variety!  Is the watering the same?

Are there enough repetitions in the project? Maybe a dozen might give you a clue if the differences are dramatic. If the differences are small you will need many more.

What about the vagaries of your pest and disease? You might eliminate slug damage from your analysis by discarding that example from your results, but is that not bias, slugs might like whatever you are testing? Worse do you eliminate the slug damage if doing so favours the result you prefer but you fail to make such an adjustment the other way round!

Your slug damage might be a black swan

You might inadvertently demonstrate something different to what you mean to. Suppose the said product really was quite useless and you had decided to work it into the ground. You would need similarly to disturb the ground in your control otherwise you would show the effect of hoeing!

Do you have an interest invested in the result? To win an argument, prove a philosophy or perhaps just to be nice to whoever gave you the product to test?

Even if you get a result for your conditions, in that season, in that weather, with that unexpected frosty night, for that time you were unable to water when you got back late, for those conditions you subconsciously thought your product would do well in when you set up the experiment, on your own soil with your own favourite variety: you cannot claim to the world you have proved anything at all.

Then how do we learn?

It gives me a lot of fun to bury newspaper. Eric Robson once filmed me burying newspaper for a TV programme!

By all means try out things different and see how it works for you. Perhaps do little tests. When I started to think about burying newspaper I checked how much water an inch of the Times newspaper would absorb! My result showed it to be substantial but I made the mistake of thinking ten times that thickness would hold ten times the water. It does not as water in deep profiles drains away more.

When my friend Peter read that the Himalayan balsam spreads when the seed floats down rivers and streams he checked out his doubts by adding the seed to a bucket of water and finding it sinks! He would of course agree that like silt, seed might be washed down the river but it is likely that most of the spread is by the well known mechanical dispersal

Mechanical dispersal does the job very well

Both the above are petty examples but do illustrate you should have a questioning mind.

The best way to learn is to be shown and taught by skilled experienced practitioners or to read books and magazines. It is also the worst way when there is so much myth and magic that comes from such sources. Gardeners who have science in their background do have an advantage when their knowledge of nature enables them to ask the right questions and sort the wheat from the chaff.

When I am out of my own area of expertise I find myself learning by listening to those I respect and following their lead to check things out further. In matters medical for the last ten years I have followed the sound common sense, independence, experience and encyclopedic knowledge of worldwide medical research by Dr. David Grimes.

When I speculate about hybridity, I freely confess that without access to the writing of geneticist Gene McCarthy I would not have been able to put pen to paper.

When it comes to gardening I wend my own way.

There are many ways to garden. Fortunately most of them work even if methods are not perfect or are even disputed. Just do it and enjoy it. Be open to new ideas and be prepared to make mistakes and learn. Soak up information wherever it comes from but always question.

Every year I try things different but I never have done a designed 'trial'. I have had my triumphs and disasters. I hope I will have many more.

I bumble along

I am trying my home made charcoal as a compost for my tomatoes this year

I hope in my zeal to question the validity of a trial that this not deter you from experimenting with things new. Suppose you are considering trying a new variety of vegetable. You might try the new one but still take a crop of the old one as insurance! The new one might prove to be much nicer but perhaps harder to grow. You will make a subjective decision about what you do in the following year!

Not much of a trial

I took blogger Mark Willis’s advice last year  and tried climbing french bean ‘Cobra’ - in my own case to grow it for an early crop in my greenhouse.

(The previous season I had done the same with runner bean Polestar. It grew and yielded very well. Unfortunately most of the beans matured at the same time and Brenda does not really like runner beans. I had chosen Polestar as a self fertile runner bean as commercial research shows in poly-tunnels tunnels there are insufficient pollinators for cross pollinated varieties. When my leaky greenhouse positively buzzed with bumble bees I realized I had not really needed to chose Polestar)

Last year I sowed six seeds of Cobra in a three inch pot in my warm conservatory on the first of April – it is my only heated growing environment and as we live in that room most of the time I am only allowed a very small corner. The germinated seedlings were planted together at a single station about mid April in my unheated greenhouse. They sulked for a while and the slugs had a go at them. When warmer weather arrived they took off to give a dozen large boilings for the two of us before the outdoor beans were ready. Cobra is a wonderful bean and it grew very well.

This year the equivalent young beans are still sulking. Will they survive until warm weather comes? It’s all trial and error!

I write these posts in advance. In this case the beans were sulking too long and I sowed some more. Now in warmer weather they are chasing each other roof wise!

The beans on the right were sown nearly a month later! In a different season the first sowing might have raced away and in another would be dead!

I never do the same things twice! My indoor sugar peas in the higher picture were eaten by the mice and the re-sowings this time in a seed tray are just starting cropping

More on commercial research

I started this post by saying that any attempt to do real research and to do a genuine trial by gardeners such as me is a complete waste of time as the results will never be statistically valid and will always be seriously flawed rendering them worthless. I wonder how much we should trust ‘real research’.

Perhaps not very much! As I have hinted commercial research is not really geared to the very different needs of we amateurs. Commercial growing is a completely different ball game.

Horticultural research is not done for amateurs

We live in a society where researchers are paid by results. We live in a world where vested interests want to persuade you. We hear from the media much conflicting information based on so called research.

Trials can be designed in such a way as to give almost predictable conclusions that will serve a vested interest. Researchers are not required to publish negative results they don’t like. So called ‘black swans’ are rejected. Neutral conclusions go unreported because they are not very sexy. Worse some research receives world wide publicity and yet is based on very few replications and has never been reproduced by a disinterested party.

The kind of horticultural research about which I am particularly doubting are those trials that are exclusively laboratory based. I am particularly dubious when chemicals are directly applied to a plant or an animal when testing for safety in use. Things need to be verified in real field situations.

I am also suspicious where genome studies on plants are used predictively without really growing them.

I asked scientist friend Peter which research I should respect. He replied that I should enquire who is paying.

One final irony. The most dubious claims for new ‘wonder products’ are made without any research at all. The actual facts get in the way

Links

David Grimes in full flood

Gene McCarthy writes about how new hybrids stabilise

I bury newspaper

My own trial lacked credibility

My post about black swans had some of Cathi’s very nice pictures