marktforschung.de
- succeet.de
- Blog
- Recognising sentiments with AI text analysis – new approaches in social media research
Interview with Marc Trömel, VICO Research & Consulting – Cooperation Partner succeet22 Recognising sentiments with AI text analysis – new approaches in social media research
succeet: AI and web monitoring have been around for a while and are already being used in market research and quality management. What are new approaches and possibilities?
Marc Trömel: Many technologies have been known for a long time. However, it is the combination and the right use of the technologies that make the difference.
succeet: What does this mean exactly?
Marc Trömel: Let’s say you want to know what is good or bad about your product. You are interested in how you benchmark towards the competition and what problems consumers have. You want to improve your product and market it properly. Now, in the automotive sector, for example, you are confronted with about 10 million texts written in a month about your and your competitors' vehicles by consumers in different countries. Almost everything you need to know about most vehicles is available. But the amount of data is too big. You might be able to read a small sample on your own vehicle, or have it read by your service provider, but you can't get the whole picture.
succeet: What is your approach solving this? How do you achieve a valid benchmark analysis?
Marc Trömel: It's a multi-step process. First, you must find everything about the vehicles. That's not easy, because often people write about the vehicle, but the vehicle is not mentioned in the text at all. Then you have to get rid of irrelevant things like advertising, statements by journalists or news to be able to evaluate only the pure consumer opinions. You also don’t want to evaluate quotes in order to avoid falsification of results by repetitions.
Then the texts have to be analyzed automatically. First, it must be identified where in the text is a statement about the vehicle at all. Then it must be found where problems or positive characteristics of the vehicle are mentioned. Vehicle recognition is automated using search queries, artificial intelligence and knowledge databases. Then, the correct vehicle is assigned to the correct statement. Afterwards it must be recognized whether the statements are written in a negative or positive way. AI is used here as well.
succeet: So you have developed a process that allows you to extract very specific statements from social media, based on a very large amount of data. Can you give us an example?
Marc Trömel: Let's look at a text from a SEAT Ibiza forum: "I recently got the new winter tires from Goodyear. In the past two weeks I had a flat tire twice. This could be coincidence, but I think it's due to the quality." The system recognizes: SEAT Ibiza/tires/negative, Goodyear/Image/negative.
succeet: How does the system do this?
Marc Trömel: It has a lot of knowledge about language, of course, it knows all vehicles and companies, and in this case it was trained specifically for the automotive industry.
succeet: How close to human quality are we?
Marc Trömel: Partly above, partly below. Especially regarding the sentiment the AI does make mistakes (90% correct), but much fewer mistakes than a human would make. It's similar for vehicle recognition (96% correct). In recognizing the correct subject, the machine achieves a precision of just over 70%.
succeet: Why only 70%?
Marc Trömel: We track about 800 topics in the automotive industry. The topics are quite detailed and sometimes difficult to distinguish from each other. If you aggregate the topics to topics like interior or quality, then you have a much better quality, which is around 85%.
succeet: Those are good results! Do you have a few more facts and figures for us?
Marc Trömel: We are currently tracking 32 brands and 8,000 vehicles in the automotive environment. To bring our system to this level, we have manually classified over 500,000 texts and used them as training data.
succeet: How easily can the methodology be transferred to other industries?
Marc Trömel: The methodology is quite quickly transferable as well. We have similar approaches being implemented in other industries. You have to remember that we have been classifying data for 17 years now. We can build on some data here.
Thank you very much for the interview! I am looking forward to the presentation and exciting conversations at succeet22.
