Nicholas Carlini

Nicholas Carlini is a researcher affiliated with Google DeepMind who has published research in the field of computer security and machine learning. He is particularly known for his work on adversarial machine learning.

Education

Nicholas Carlini obtained his Bachelor of Arts in Computer Science and Mathematics from the University of California, Berkeley in 2013. He then continued his studies at the same university, where he pursued a PhD under the supervision of David Wagner, completing it in 2018.

Research

Nicholas Carlini is known for his work on adversarial machine learning. In 2016, he working alongside David Wagner to develop the Carlini & Wagner attack, a method of generating adversarial examples against machine learning models. The attack was proved to be useful against defensive distillation, a popular defense mechanism where a student model is trained based on the features of a parent model to increase the robustness and generalizability of student models. The attack gained popularity when it was shown that the methodology was also effective against most other defences, rendering them ineffective. In 2018, Carlini, demonstrated a attack against Mozilla Foundation's DeepSpeech model where he showed that by hiding malicious commands inside normal speech input the speech model would respond to the hidden commands even when the commands were not discernible by humans. In the same year, Carlini and his team at UC Berkeley showed that out of the 11 papers presenting defenses to adversarial attacks accepted in the year's IEEE Symposium of Security and Privacy, seven of the defenses could be broken. More recently, he and his team have worked on large-language model creating a questionnaire where humans typically scored 35% whereas AI models scored in the 40 percents, with GPT-3 getting 38% which could be improved to 40% through few shot prompting. The best performer in the test was UnifiedQA a model developed by Google specifically for answer question and answer sets.