Meet the Team #2: Lingjuan, Jerone and Roberto

What do privacy, pattern recognition, and percussion all have in common? They are concepts and creative endeavors that have inspired Sony AI team members Lingjuan, Jerone and Roberto. Read on to learn more about these three Sony AI team members.

What is your area of expertise?
My main research areas lie at the intersection of privacy, security and machine learning. Although deep neural networks (DNNs) have been widely applied in computer vision, natural language, and robotics, DNNs face a series of privacy and robustness issues. I've worked in these areas for over five years and will be dedicated to helping solve privacy and robustness issues in real-world AI systems for my whole life.

What made you interested in working in AI or your area of expertise?
I believe AI can help revolutionize our life, and privacy and robustness are two challenging problems that may hinder the development of AI. Without privacy and robustness, AI may lose the trust of consumers, and become out of human control in real applications. My journey in this area began in 2015 when I was still a Ph.D. candidate in University of Melbourne. I am very optimistic about privacy-preserving machine learning, especially collaborative/federated learning, and found this new learning paradigm is very promising. It offers a privacy-aware learning paradigm that does not require raw data sharing but allows participants to collaboratively construct a better global model by only sharing less sensitive information. Nowadays, federated learning has attracted significant attention, and has been widely used in many industrial applications. I am excited to witness the flourishing of my research area and feel it is a personal accomplishment that I chose it as my research topic. Later on, I noticed that although Deep Neural Networks (DNNs) are very useful, they are also very fragile, particularly vulnerable to both poisoning and evasion attacks. I feel it’s urgent to help safeguard DNNs to defend against these attacks and build more robust models to benefit humans.

What made you excited about joining Sony AI and what do you hope to accomplish?
Sony AI has great potential to help unleash human imagination and creativity. I hope to bring privacy and security capabilities to the products of Sony to benefit the billions of people that use Sony products. To accomplish this goal, I’m currently building a Privacy-Preserving Machine Learning (PPML) Team in Sony AI, and aiming to attract more talented people to the team to help us build privacy and robustness into Sony products.

Dr. Lingjuan Lyu is currently a senior research scientist and team leader in Sony AI. She was an expert researcher in Ant Group, Research Fellow in National University of Singapore, and Research Fellow (Level B3, same level as lecturer/assistant professor) in Australian National University. She received Ph.D. from the University of Melbourne. She was a winner of the IBM Ph.D. Fellowship Worldwide and contributed to various professional activities, including ICML, NeurIPS, AAAI, WWW, etc. Her current research interests span federated learning, AI privacy and security, fairness, edge intelligence, etc. She has published over 40 papers in top conferences and journals, including NeurIPS, ICLR, IJCAI, SIGIR, etc. Her paper won the best paper award in FL-IJCAI’20.

What made you interested in working in AI or your area of expertise?
As a child, I attended a Montessori primary school in London; one of my earliest memories was being fascinated by the ‘Pink Tower’ (a series of cubes designed as a sensorial learning tool), which I attribute to my early interest in pattern recognition. This initial interest has never left me and led me to study mathematics at King’s College London.

During my studies, particularly while completing my final year dissertation, I became increasingly interested in the field of machine learning. At the time, I was especially curious about how it was being used to tackle significant real-world problems. I subsequently completed a PhD in computer science, within the Vision & Imaging Science group, at University College London.

What made you excited about joining Sony AI and what do you hope to accomplish?
In addition to Sony’s preeminent reputation in the field of technology, what initially drew me to this role was the opportunity to join Sony AI at its inception. I was particularly excited by the prospect of working collaboratively with colleagues to shape the direction of the research.

As an avid cook, I was thrilled to see the Gastronomy flagship project listed. Much like AI research, one of the things that I like the most about cooking is that it fosters creativity through experimentation. Moreover, I was especially pleased to see Sony AI’s firm commitment to the examination of the scientific, social and ethical impacts of AI algorithms, through its AI Ethics flagship project. As an innately interdisciplinary field, I believe that AI ethics impels one to adopt a more in-depth approach when considering the practical implications of the development and deployment of AI. The activities of this team are far-reaching and I hope we will be able to shape the future of Sony as a whole. For instance, by creating guidelines and policies that can help to set a standard on the use of AI.

There are many ways that people express creativity. What is your favorite form of creative expression?
Visual communication is probably my favourite form of creative expression, primarily due to its ability to embody concepts and transcend language. I’m a big fan of the book ‘Grid Systems in Graphic Design’ by Josef Müller-Brockmann. It’s essentially an exposition on how to create aesthetically pleasing modular layouts, which was initially developed by a group of Swiss graphic designers who were inspired by typographical literature to create a more rigid and coherent system for page layout. While I don’t always adhere to the guiding principles, I often refer to it when making presentations.

Jerone is a Research Scientist and contributes to the Sony AI Gastronomy and AI Ethics flagship projects. His research interests and expertise span computer vision and deep learning, in particular self-supervised anomaly detection, transfer learning, and adversarial machine learning. Prior to joining Sony, Jerone received an MSci in Mathematics from King’s College London, which he followed with an EPSRC-funded MRes and PhD in Computer Science at University College London (UCL). Subsequently, Jerone was awarded a Royal Academy of Engineering Research Fellowship, a British Science Association Media Fellowship with BBC Future, and a Marie Skłodowska-Curie RISE grant. While at UCL, Jerone also spent time as a Visiting Researcher at the National Institute of Informatics (Tokyo) and Telefónica Research (Barcelona).

What made you interested in working in AI or your area of expertise?
While studying ancient Greek, Latin, and Italian Literature in my humanistic high-school curriculum, I became fascinated with programming and what computers enable. It all started as a hobby: I spent more and more time coding, learning about computers, and understanding their inner mechanisms. While all of this became my main activity as a University student, I still had the desire to understand the human mind, its creativity, and the results of intelligence, whose maximum expression is - from my perspective - art in all its forms. This generated in me the desire to understand how computers can learn and become intelligent machines that, by operating in a physical world, can also produce beauty and help humans in doing that.

What made you excited about joining Sony AI and what do you hope to accomplish?
The idea that human creativity and human life can be extended, augmented and improved by means of AI. I hope to be able to make an (indirect) impact in people's life by means of my research.

What is your favorite form of creative expression? What most inspires you to be creative?
Figurative arts and music are the most relevant creative expressions in my life. I grew up playing drums - and percussion in general, while being part of an orchestra in my hometown, Itri. There, I started painting and drawing, and I also learned how to make big drawings with flowers on the street. Finally, poetry is the way I express my emotions with words. This is very important, as it enables me to relax, reset my mind, and get a fresh perspective on everything: either at work or outside work.

Roberto is a Research Scientist at Sony AI and external Professor at Sapienza University of Rome. Previously, he was a Research Scientist at Cogitai and an Assistant Professor at Sapienza University of Rome, where he also received his PhD, under the supervision of Prof. Daniele Nardi. Roberto was a visiting Research Scholar at the Robotics Institute - Carnegie Mellon University (Pittsburgh, USA), under the supervision of Prof. J. Andrew Bagnell. He has been awarded a 3-years fellowship, a Research Starting Grant by Sapienza University of Rome, and a Robotics Fellowship in 2016 by AAAI. Roberto completed his MSc in AI and Robotics in 2013, at Sapienza University of Rome, and his BSc in Computer Engineering at University of Pisa in 2011.