Detecting The Corruption Of Online Questionnaires By Artificial Intelligence

Our recent paper on detecting the corruption of online questionnaires by artificial intelligence was recently published in the Frontiers In Robotics and AI Journal. We created a short explainer video about our project:

Here is the abstract of the paper:

Online questionnaires that use crowd-sourcing platforms to recruit participants have become commonplace, due to their ease of use and low costs. Artificial Intelligence (AI) based Large Language Models (LLM) have made it easy for bad actors to automatically fill in online forms, including generating meaningful text for open-ended tasks. These technological advances threaten the data quality for studies that use online questionnaires. This study tested if text generated by an AI for the purpose of an online study can be detected by both humans and automatic AI detection systems. While humans were able to correctly identify authorship of text above chance level (76 percent accuracy), their performance was still below what would be required to ensure satisfactory data quality. Researchers currently have to rely on the disinterest of bad actors to successfully use open-ended responses as a useful tool for ensuring data quality. Automatic AI detection systems are currently completely unusable. If AIs become too prevalent in submitting responses then the costs associated with detecting fraudulent submissions will outweigh the benefits of online questionnaires. Individual attention checks will no longer be a sufficient tool to ensure good data quality. This problem can only be systematically addressed by crowd-sourcing platforms. They cannot rely on automatic AI detection systems and it is unclear how they can ensure data quality for their paying clients.

Peer Review Review

In October 2023 I had the privilege to talk at the Nerd Night in Christchurch. This event series operates at the intersection of comedy, popular culture and science. I talked about my adventures in exploring the peer review process. Some of them to the annoyance of my fellow scientists, conference organizers and predatory publishers. But always with a nod to comic effect and a focus on the overcompetitive beast we call academia.

I also published this event on the HRI Podcast.

Jibo is dead (again)

In 2020 I recorded a podcast episode entitled “Why do all social robots fail in the market?“. I interviewed Tomas Concha from NTT Disruption, the company that had bought the commercially unsuccessful robot Jibo. I already had my doubts about NTT Disruption in 2020. In 2023 NTT Disruption was disrupted. Meaning that it closed down and with it Jibo. This does seem to be the end for this little useless robot.

But don’t worry! Other companies continue to build largely useless robots that are not much more than smartphones on wheels. Have a look at Samsung’s Ballie robot.

The idea of a smart home robot is not new. Amazon developed their Astro robot, but did not sell it to the general public. LG is also presenting a robot at CES2024 with roughly the same features.

I wonder if Samsung or LG will sell their robots to consumers. Or is this just another robot PR gag?

Posters of famous computer scientists

There are many important computer scientists and innovators. I created a small and very personal list and created posters to honour their contributions.