Evaluating coherence in AI-generated text

Abstract

[EN] This study examines the role of coherence in AI-generated online reviews and its effect on perceived authenticity and consumer trust. By applying advanced metrics like BERT Score, BART Score, and Disco Score, the research analyzes the coherence of AI-generated text using Generative AI models, specifically Llama-2, on Amazon beauty product reviews. Results indicate that AI-generated reviews exhibit higher coherence compared to human-generated content, suggesting that Generative AI can produce seemingly authentic content. This finding challenges the ability to distinguish between human and AI-generated reviews, raising important questions about consumer trust in digital marketplaces. The study underscores the importance of coherence in online content's credibility and opens avenues for further research on Generative AI's role in e-commerce.