Table of Contents
- Introduction / Foreword
- Short facts about the eology AI content study
- Study description: what, when, how, who? Procedure and more
- Hypothesis 1-4
- Learnings and recommendations
- Conclusion and outlook
Despite all the positivity surrounding AI texts, there are still a few final words to ponder: what will happen when the majority of content on the web is created solely by AI? Won’t more and more humanity be lost? And what about the spread of misinformation? This is possible at an alarming speed with the help of AI. We should all bear this in mind when we use chatbots. It is up to us to operate, control and, in case of doubt, even regulate AI.
Sample chapter
Study description: What, When, How, Who? Procedure and more
During the test period from 1 August to 30 September 2023, the eology AI task force tested 134 use cases and thus obtained qualified results on AI text creation. The texts created by the chatbots were evaluated and assessed and rated in comparison to purely human-generated texts.
Note
The term “humanly created” requires a definition that creates clarity. After all, not everyone has the ability (knowledge of the German language, editorial training, etc.) and the talent (affinity and talent for writing) to create high-quality texts. In the case of the eo AI content study, there is also the aspect of SEO knowledge.
For this reason, the AI texts in the eology AI study are compared with texts written by trained SEO editors and evaluated according to the quality standard that eology Content Creation follows for high-quality value-added content with high ranking potential.
The test results serve to prove or disprove the 4 hypotheses on content creation with artificial intelligence. Furthermore, the eology AI task force has derived recommendations for the use of chatbots in SEO text production. They serve companies as guidelines for content creation with the use of AI.
The AI Content Study: Answers to the Most Important Questions
Can AI write good SEO content?Yes, with some caveats. In the eo AI Content Study, chatbots like ChatGPT achieved surprisingly good results in terms of language and met the defined quality standard of 75 percent across many criteria. You can rely on AI, especially when it comes to language quality – that is, grammar, spelling, and clarity. However, the texts are weaker when it comes to targeting specific audiences and, to some extent, pure SEO optimization. Therefore, texts ready for publication still require manual review and editing.
Can AI-generated texts be
distinguished from texts written by humans?Hardly. In the study’s blind tests, even trained SEO copywriters found it difficult to reliably identify AI-generated text. The accuracy rates varied widely, and not a single test subject correctly identified all versions. Supposed AI markers, such as rigid structures or unusual phrasing, sometimes even led to incorrect conclusions, because humans make such mistakes as well. The assumption that AI-generated texts can be clearly identified has thus not been confirmed.
What are the key considerations when
using AI to create text?The most important factor is the prompt, true to the saying “garbage in, garbage out.” The more specific and detailed you make the task and the more relevant information you provide to the AI, the better the result will be. A proven prompt setup includes a clear task, context (what role the AI should play), and modification (the target audience for the text). Good training texts used as templates further improve the output. Standard prompts, on the other hand, are rarely sufficient.
Will AI Replace the SEO Copywriter?As things stand, no. AI takes many tasks off the hands of experienced copywriters and significantly speeds up production, but humans remain indispensable for high-quality content that meets E-E-A-T standards. Chatbots lack personal experience, emotional intelligence, and the ability to reliably evaluate sources and facts. AI works best as a sparring partner: it provides the foundation, while humans ensure quality, uniqueness, and genuine added value.
The important thing here is:As this is an empirical study based on a limited text corpus with the aim of assessing SEO performance potential, other methods and other test scenarios and use cases may produce different results. The eology AI task force welcomes constructive feedback, professional exchange and discussions.