A new study claims of the „Agriculture Industry Association“ tries to proof that organic farming is less sustainable than conventional farming. This finding is in sharp contrast to the published literature and the study in fact has a lot of shortcomings, which are worth commenting.
In Germany, the beginning of January is usually time for the so called ‘green week’ (“Grüne Woche”), a fair in Berlin, where agricultural producer and the agribusiness present their products. During this green week, the different farmers associations and environmental NGO are coming up with their political proposals. So on January 14, 2016, the German ‘Agriculture Industry Association’ (Deutscher Industrieverband Agrar (IVA)) presented a study showing that ‘productive agriculture causes less biodiversity-loss’. The study was basically a yield- and biodiversity-comparisons between organic and conventional farms, which I want to put into perspective here.
The main results (based on bookkeeping data of German farms) is that organic farms achieves a lower yields and also lower yield-growth than conventional farms. First of all, this is not a new or surprising results, since these results from the German test farm-network are yearly reported by the Ministry of Agriculture. However, the author of the study Dr. Steffen Noleppa used the data-set to analyse the yield-performance of different crops. The following graph taken from the study shows the yield-growth in two periods, between 2007/08 to 2009/10 and between 2011/12 to 2013/14:
The graph documents, that organic farms had lower increases of yields in both periods and that the yield gap of between organic and conventional farms is drastically increasing. Showing those data, we need to discuss, whether bookkeeping data are reliable enough to compare yields. There are some arguments in favour and against this database and sadly enough, none of these arguments have been mentioned or discussed by Steffen Noleppa, which is my first criticism on this study. Anyhow, let us start with some arguments in favour of bookkeeping data:
- The database of test farms (usually hosted by Thünen Institute in Braunschweig) is one of the broadest and most detailed database. Therefore, if we want to have a broad and representative source for yield comparisons with many observations, this data-source is a good option.
- The author of the studies, the agricultural economist Steffen Noleppa from ‘HFFA Research’ has chosen farms with a similar soil-quality index on the farm level, which is an appropriate and established method to compare organic and “comparable conventional farms”. A similar method is used by the Thünen Institute for economic comparisons of organic and conventional farms.
Therefore, this database can provide empirical results based on a broad data-set. The message of Noleppa however is not brand-new, we can observe the same long-term trend since many years. I have used the same data-set to do exactly the same type of comparison as in the following graph published in one of my post on efficiency and productivity of organic farms, but mentioning the short-comings of this type of data. If we compare this data-source with e.g. field experiment, there a number drawbacks of book-keeping data:
- Errors in yield-data: The main purpose of bookkeeping data in Germany is the tax-declaration. Yields are recorded not in the obligatory, but in the voluntary part of the bookkeeping. According my experience with bookkeeping data, we find more error-values in the voluntary than in the obligatory part. But besides typos and incomplete records, I am not 100% sure, how internal use of grain is treated in this part of the bookkeeping. This should be recorded, but this is not easy, since the internal use of fodder-grain might not be measured precisely. I assume that those problems are being corrected in the before mentioning the data. Thünen Institute has a long experience with this type of data, however the description of methods and database in the study is quite short without any background.
- Use of grain is unclear. A huge part of the grains in organic farming are used as fodder-grain. Farms keeping animals are producing their own feedstuff, therefore, we might observe a higher rate of fodder-grains as in conventional farming. There are some indication according Agrarmarkt Info GmbH, that about 65% of all organic grains produced in Germany are used as feedstuff. However, this is a rough estimate and comparisons to conventional farming (with 62%) might be misleading, since there are no restrictions for conventional animal farms to by external feedstuff. This system-difference might also cause part of the yield-gap. And again: No mention in the study of Noleppa.
- Locations of organic farms: In Germany many organic farms work in regions with low yield potential. The comparison of Noleppa takes this into account on the farm level, however, the yield potential of the single field is not known and therefore not used in the study (in contrast to the press-release).
So in a nutshell, we don’t know to what extent bookkeeping data is reliable for this type of study. Therefore, field experiments will bring more precise results than bookkeeping data, however bookkeeping data have the broader empirical basis and reflect the sectoral results ‘as the are’. We should discuss findings from these data, but I would argue, that it is highly problematic to estimate something like the ‘world-production’ based on German bookkeeping data. For projection, field trials or practical yield assessments might be more appropriate.
There has been the first meta-study of Badgley et al. (2007) “Organic agriculture and the global food supply”, who found a yield ratio in grain production of 69 per cent (Badgley et al. 2007). They also compared industrialized and developing countries and found on average even higher yields. The study of Bagdley et al. (2007) has been criticised, since according Avery (“Organic abundance’ report: fatally flawed”), they used a too broad und partly unclear definition of organic farming (Avery 2007). However, they started a complete string of literature. Some years later, de Ponti et al. (2014) found organic yields in cereal production to be on average 79 per cent of the conventional yield-level, but huge range from 40 to 145 per cent (de Ponti et al. 2012). Seufert et al. (2012, Comparing Yields of organic and conventional Agriculture) found a yield ratio of 75 per cent, Ponisio et al. (2014) found (using a more strict method of study-selection) a yield ratio of 80.8 per cent. They excluded ‘subsistence yields of unimproved agriculture’ (Ponisio et al. 2014). This paragraph shows, that most studies find a yield level between 70 and 80 per cent of the conventional level, but with a wide range of results (as it is shown in figure 3 from the study of de Ponti et al. 2014).
Steffen Noleppa is mentioning some of the studies, however without a fair discussion of the results.
There is a second part of the study, which is even more critical. Noleppa asks, whether conventional farming would provide more biodiversity in relation to their high yield levels by comparing the relation of biodiversity and yields. The main result of Noleppa is, that based on the yield differences, conventional farming is loosing biodiversity by 86 per cent and organic farming by 67 per cent.
I see the main problem again in the selectivity of literature. And at this point, the study losses it’s scientific relevance: There is a long chapter discussing literature on biodiversity, without taking into account a number of important studies, published in international journals. Just to mention one recent study from Schneider et al. (2014) published in Nature in June 2014, also coming to significant advantage for organic farming systems. I won’t list all the other studies from the past yeards, which find a substantially higher biodiversity on organic farms. There is a rich stream literature, which is simply ignored by Noleppa. The most prominent point in Noleppas study is, that there are also cases, where conventional farms have the same or even better levels of biodiversity. So the exemption is emphasized and the regular case hardly mentioned, that organic farming has a higher level of biodiversity. We also learn things like ‘the one biodiversity does not exist from an objective point of view and its measurement is more or less a subjective matter’. So obviously Noleppa concludes, that we might be able to change (or even manipulate?) the level of biodiversity by simply changing the settings of a trial. This is a strong statement against ecology research in general, which is mostly published in peer reviewed scientific journals.
The same holds for Noleppas selection of biodiversity-studies: Many of the single figures are difficult to verify according the literature, which is the case for the study Armentgot et al. (2011). I still wonder how Noleppa calculated the biodiversity-loss values, none of the values used by Noleppa are directly mentioned in the study. Another study from Reidsma et al. (2006) is cited with positive figures for conventional farming. However, this study is a theoretical modelling-study, which does not use empirical data. The figures for biodiversity loss were assigned to the farming system before modelling in order to see, what happens if those figures are assigned. So presenting this study as empirical is strongly misleading. The source of Keeling and Lillywhite (2012) is a poster from University of Warwick. This is not an argument against scientific posters and the results look credible and well documented. However, the figures cited by the study of Noleppa do not appear at all on the poster, for whatever reason. The study of Lillywhite et al. 2012 is much better documented, their approach also ‘assigns’ biodiversity values to the farm-types and they use exactly the same approach as Reidsma et al (2006) and Alkemade et al. (2009). And again, no empirical but rather theoretical data are presented with a totally different purpose. So Noleppa basically replicates an approach four times, which is not appropriate to compare biodiversity loss since it is not based on empirical data.
And again, there is a part of the literature missing in Noleppas study. There are some studies, which are simultaneously analysing ecological and economic data, like Eltun et al. (2002), Poudel et al. (2002) or Kantelhardt et al. (2009), to name just a few examples, where modelling has been done on real-world data, and not on theoretical data. And in two of these three studies, organic farming has advantages against the other systems. And to avoid misunderstandings: The theoretical modelling is useful, however, we should not confuse models used in Reidsma et al. (2006), with studies using empirical data.
Another approach is to relate yields to e.g. energy input: The FIBL in the 21-year DOK-trial (published 2002 in Science) found the organic yield 20% lower than the conventional farm-system. The energy-input were lowered by 34 to 53% and pesticide input by 97%. So from an efficiency-point of view, the assessment of Mäder et al. (2002) based on 21 years come to the opposite result and these are results published in science, not by the German Agricultural Industry Association. Aldanondo-Ochoa et al. (2014) is using empirical data to model productivity by using economic and environmental data (nitrogen and pesticide). According their results, the environmental productivity is 8 per cent higher for organic farms than for conventional farms (Aldanondo-Ochoa et al. 2014). Again, there are two studies using empirical data with a completely different result.
So to conclude, unfortunately the results from the test-farm network are not used to start a constructive debate, but rather constructing arguments against the organic system itself and stimulating an attack against organic farming in Germany and the Green Party, who is backing the support of organic farming. Usually, this type of debate comes at the costs of practical organic farmers! This also reflects the general policy of the „Agriculture Industry Association“, who seem to care more about the input-industries (fertilizer, plant protection…) than about practical farmer. Even the German Farmers Association (Deutscher Bauernverband e.V.) has released a press-release in December 2015 how to support organic farm, which also recognized the services of this farming system.
Yes, we need to take the yield results serious, in order to improve yield-levels in organic farming. Besides this, it would be helpful to figure out, to what extent are the bookkeeping data reliable. And finally, we cannot ignore the environmental benefits of organic farming, which are given in the literature. And this is my main criticism on Steffen Noleppa: His selection of studies leads to a distorted view on the organic farming system. There are economic and ecological benefits to the systems, which might even provide chances for developing countries and which are simply ignored by Noleppas study. A recent paper with the title “Organic Agriculture in the Twenty-First Century”, recently published in Nature Plants on February 03, 2016 by John Renagold and Jonathan Wachter is giving a good overview on the literature and their conclusions are rather positive.
Three conclusions from my part on the issue of yields and biodiversity:
- We should take the yield-gap serious. The data from Noleppa are there and this is still the constructive part of his study reminding us on this huge difference. However his explanations are not convincing and he is (at least in my view) not doing a fair system-comparison. But maybe the remarks are a starting point to better analyse the data, also giving arguments to support-programs for the specific breeding of organic varieties.
- For the world food situation, there is the necessity to consider high-yield strategies for the developing countries. At the moment, the main problem is distribution of food, but that might be different in 20-30 years from now and organic farming will no solve the quantity problem. I am not claiming to be an expert on these issues, but the study of Noleppa is not helping on that issue, since his study is not applied to the specific challenges of developing countries, but rather simply forecasting from the specific German data to the world food-situation, which is (again in my view) not a good idea.
- But there are also ecological challenges, just take the topic of land-degradation documented in a publication of IFPRI. For such farm- and ecosystems in developing countries we need to gain more insights, how organic farming works in developing countries. As Renagold concluded in 2012 in his Nature-article “Comparing Apples and Oranges”: “No one of these systems alone will produce enough food to feed the planet. Rather, a blend of farming approaches is needed for future global food and ecosystem security. Organic farming provides multiple sustainability benefits, and Seufert and colleagues‘ findings indicate that it can play a part in feeding the world. Yet just under 1% of agricultural land worldwide is now managed organically. This percentage should be much larger in the future.
As always, questions and comments are of course welcomed!