AI Content study

AI Content study

Request free whitepaper

Download our free white paper with the results of our “Content AI study” now. Our eology AI task force describes how our AI content study was conducted and analysed. You will also find learnings and recommendations in this white paper that you can use for yourself and your company.

Table of Contents

Despite all the positivity surrounding AI texts, there are still a few final words to ponder: what will happen when the majority of content on the web is created solely by AI? Won’t more and more humanity be lost? And what about the spread of misinformation? This is possible at an alarming speed with the help of AI. We should all bear this in mind when we use chatbots. It is up to us to operate, control and, in case of doubt, even regulate AI.

Dr. Beatrice
Eiring
, Head of Content Creation b.eiring@eology.de +49 9381 5829015

Sample chapter

Study description: What, When, How, Who? Procedure and more

During the test period from 1 August to 30 September 2023, the eology AI task force tested 134 use cases and thus obtained qualified results on AI text creation. The texts created by the chatbots were evaluated and assessed and rated in comparison to purely human-generated texts.

Note

The term “humanly created” requires a definition that creates clarity. After all, not everyone has the ability (knowledge of the German language, editorial training, etc.) and the talent (affinity and talent for writing) to create high-quality texts. In the case of the eo AI content study, there is also the aspect of SEO knowledge.

For this reason, the AI texts in the eology AI study are compared with texts written by trained SEO editors and evaluated according to the quality standard that eology Content Creation follows for high-quality value-added content with high ranking potential.

The test results serve to prove or disprove the 4 hypotheses on content creation with artificial intelligence. Furthermore, the eology AI task force has derived recommendations for the use of chatbots in SEO text production. They serve companies as guidelines for content creation with the use of AI.

The AI Content Study: Answers to the Most Important Questions

Can AI write good SEO content?

Yes, with some caveats. In the eo AI Content Study, chatbots like ChatGPT achieved surprisingly good results in terms of language and met the defined quality standard of 75 percent across many criteria. You can rely on AI, especially when it comes to language quality – that is, grammar, spelling, and clarity. However, the texts are weaker when it comes to targeting specific audiences and, to some extent, pure SEO optimization. Therefore, texts ready for publication still require manual review and editing.

Can AI-generated texts be
distinguished from texts written by humans?

Hardly. In the study’s blind tests, even trained SEO copywriters found it difficult to reliably identify AI-generated text. The accuracy rates varied widely, and not a single test subject correctly identified all versions. Supposed AI markers, such as rigid structures or unusual phrasing, sometimes even led to incorrect conclusions, because humans make such mistakes as well. The assumption that AI-generated texts can be clearly identified has thus not been confirmed.

What are the key considerations when
using AI to create text?

The most important factor is the prompt, true to the saying “garbage in, garbage out.” The more specific and detailed you make the task and the more relevant information you provide to the AI, the better the result will be. A proven prompt setup includes a clear task, context (what role the AI should play), and modification (the target audience for the text). Good training texts used as templates further improve the output. Standard prompts, on the other hand, are rarely sufficient.

Will AI Replace the SEO Copywriter?

As things stand, no. AI takes many tasks off the hands of experienced copywriters and significantly speeds up production, but humans remain indispensable for high-quality content that meets E-E-A-T standards. Chatbots lack personal experience, emotional intelligence, and the ability to reliably evaluate sources and facts. AI works best as a sparring partner: it provides the foundation, while humans ensure quality, uniqueness, and genuine added value.

Would you like to discuss this with our content experts?

Contact directly
The important thing here is:

As this is an empirical study based on a limited text corpus with the aim of assessing SEO performance potential, other methods and other test scenarios and use cases may produce different results. The eology AI task force welcomes constructive feedback, professional exchange and discussions.