Sony AI Reveals New Research Contributions at NeurIPS 2023

Sony Group Corporation and Sony AI have been active participants in the annual NeurIPS Conference for years, contributing pivotal research that has helped to propel the fields of artificial intelligence and machine learning forward. These contributions have included — but are certainly not limited to — a vision-based control algorithm for expert human-level driving control in a racing simulation game; metrics for measuring bias amplification; and a novel one-shot federated learning framework.

This year, at NeurIPS 2023 in New Orleans (December 10-16), the Sony AI team will be unveiling more than a dozen new exciting pieces of research related to ethical data collection, privacy, federated learning, sound events, knowledge graph embeddings (KGE), and more. See a listing of our newest contributions below.

If you are attending NeurIPS 2023, don't miss our oral presentations and poster sessions. Make sure you also drop by the Sony AI booth to say hello and learn more about our work.

For more information about Sony AI’s participation, visit: https://research.sony/conferences/neurips2023/

Sony AI Oral Presentations

• Ethical Considerations for Responsible Data Curation
• Presenter: Jerone Andrews (Sony AI)
• Authors: Jerone Andrews (Sony AI), Dora Zhao (Sony AI), William Thong (Sony AI), Apostolos Modas (Sony AI), Orestis Papakyriakopoulos (Sony AI), Alice Xiang (Sony AI)
• Date & Time: Thursday, December 14, 11:30 - 11:45 a.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1602

Sony AI Invited Keynote

• When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions
• Speaker: Lingjuan Lyu (Sony AI)
• Date & Time: Saturday, December 16, 11:30 – 11:55 a.m. EST
• Location: Virtual

Sony AI Spotlight Papers

• Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?
• Authors: Lingjuan Lyu (Sony AI), Nidham Gazagnadou (Sony AI), Xiaoxiao Sun (Australian National University, Sony AI intern), Vivek Sharma (Sony AI), Hongdong Li (Australian National University), Liang Zheng (Australian National University)
• Date & Time: Wednesday, December 13, 6 - 8 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1714

Sony AI Poster Presentations

• Where Did I Come From? Origin Attribution of AI-Generated Images
• Authors: Lingjuan Lyu (Sony AI), Zhenting Wang (Rutgers University, Sony AI intern), Chen Chen (Sony AI), Yi Zeng (Virginia Tech, Sony AI intern), Shiqing Ma (UMass Amherst)
• Date & Time: Tuesday, December 12, 11:45 a.m. - 1:45 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1624

• UltraRE: Enhancing RecEraser for Recommendation Unlearning via Error Decomposition
• Authors: Lingjuan Lyu (Sony AI), Yuyuan Li (Zhejiang University), Chaochao Chen (Zhejiang University), Yizhao Zhang (Zhejiang University), Weiming Liu (Zhejiang University), Xiaolin Zheng (Zhejiang University), Dan Meng (OPPO Research Institute), Jun Wang, (OPPO Research Institute)
• Date & Time: Tuesday, December 12, 6:15 - 8:15 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1520

• Is Heterogeneity Notorious? Taming Heterogeneity to Handle Test-Time Shift in Federated Learning
• Authors: Lingjuan Lyu (Sony AI), Weiming Zhuang (Sony AI), Yue Tan (University of Technology Sydney, Sony AI intern), Chen Chen (Sony AI), Xin Dong (Sony AI), Guodong Long, (University of Technology Sydney)
• Date & Time: Thursday, December 14, 11:45 a.m. - 1:45 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1126

• Towards Personalized Federated Learning via Heterogeneous Model Reassembly
• Authors: Lingjuan Lyu (Sony AI), Jiaqi Wang (Pennsylvania State University),Xingyi Yang, (National University of Singapore), Suhan Cui (Pennsylvania State University), Liwei Che (Pennsylvania State University), Dongkuan Xu (North Carolina State University), Fenglong Ma (Pennsylvania State University)
• Date & Time: Thursday, December 14, 6 - 8 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1218

• STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events
• Authors: Yuki Mitsufuji (Sony AI), Kazuki Shimada (Sony AI), Archontis Politis (Tampere University), Parthasaarathy Sudarsanam (Tampere University), Daniel A. Krause (Tampere University), Kengo Uchida (Sony Group Corporation), Sharath Adavanne (Tampere University), Aapo Hakala (Tampere University), Yuichiro Koyama (Sony Group Corporation), Naoya Takahashi (Sony Group Corporation), Shusuke Takahashi (Tohoku University), Tuomas Virtanen (Tampere University)
• Date & Time: Thursday, December 14, 6 - 8 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #119

• Posthoc privacy guarantees for collaborative inference with modified Propose-Test-Release
• Authors: Abhishek Singh (MIT), Praneeth Vepakomma (MIT), Vivek Sharma (Sony AI), Ramesh Raskar (MIT)
• Date & Time: Tuesday, December 12, 11:15 a.m. — 1:15 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #2005

Sony AI Workshop Presentations

• Privacy-preserving design of graph neural networks with applications to vertical federated learning
• Authors: Ruofan Wu (Ant Group), Mingyang Zhang (Ant Group), Lingjuan Lyu (Sony AI), Xiaolong Xu (Ant Group), Xiu-Hao Deng (Ant Group), Xinyi Fu (Ant Group), Tengfei Liu (Ant Group), Tianyi Zhang (Ant Group), Weiqiang Wang (Ant Group)
• Date & Time: Friday, December 15, 10 a.m. EST
• Location: In-Person Workshop, “New Frontiers in Graph Learning (GLFrontiers),” Hall C2

• FRUNI and FTREE: Synthetic knowledge graphs for evaluating explainability
• Authors: Pablo Sanchez-Martin (Sony AI), Tarek R. Besold (Sony AI), Priyadarshini Kumari (Sony AI),
• Date & Time: Saturday, December 16, 10 a.m. EST
• Location: Hybrid Workshop, “XAI in Action: Past, Present, and Future Applications”

Affiliated Sony AI Posters

• LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
• Authors: Peter Stone (Sony AI, The University of Texas at Austin), Bo Liu (University of Texas, Austin), Yifeng Zhu (University of Texas, Austin), Chongkai Gao (Tsinghua University), Yihao Feng (Salesforce Research), Qiang Liu (University of Texas, Austin), Yuke Zhu (University of Texas, Austin)
• Date & Time: Tuesday, December 12, 11:45 a.m. - 1:45 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1414

• f-Policy Gradients: A General Framework for Goal-Conditioned RL using f-Divergences
• Authors: Peter Stone (Sony AI, The University of Texas at Austin), Siddhant Agarwal (University of Texas, Austin), Ishan Durugkar (University of Texas, Austin), Amy Zhang (University of Texas, Austin)
• Date & Time: Wednesday, December 13, 11:45 a.m. - 1:45 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1302

• ELDEN: Exploration via Local Dependencies
• Authors: Peter Stone (Sony AI, The University of Texas at Austin), Zizhao Wang (University of Texas, Austin), Jiaheng Hu (University of Texas, Austin), Roberto Martín-Martín (University of Texas, Austin)
• Date & Time: Thursday, December 14, 11:45 a.m. - 1:45 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1423

• FAMO: Fast Adaptive Multitask Optimization
• Authors: Peter Stone (Sony AI, The University of Texas at Austin), Bo Liu (University of Texas, Austin), Yihao Feng (Salesforce Research), Qiang Liu (University of Texas, Austin)
• Date & Time: Thursday, December 14, 6 - 8 p.m. EST
• Location: Main Conference, Great Hall & Hall B1+B2 #1221

Affiliated Sony AI Workshop Presentations

• Latent Skill Discovery for Chain-of-Thought Reasoning
• Authors: Peter Stone (Sony AI, University of Texas at Austin), Zifan Xu (The University Texas at Austin)· Haozhu Wang (Amazon), Dmitriy Bespalov (Amazon), Yanjun Qi (Amazon)
• Date & Time: Friday, December 15, 9:50 a.m. EST
• Location: Workshop on robustness of zero/few-shot learning in foundation models (R0-FoMo)

• t-DGR: A Trajectory-Based Deep Generative Replay Method for Continual Learning in Decision Making
• Authors: Peter Stone (Sony AI, University of Texas at Austin), Bo Liu (University of Texas, Austin), William Yue (University of Texas, Austin)
• Date & Time: Friday, December 15, 10 a.m. EST
• Location: Neurips workshop on Agent Learning in Open-Endedness (ALOE)

• Efficient infusion of self-supervised representations in Automatic Speech Recognition
• Authors: Darshan Prabhu (Sony Research India), Saiganesh Mirishkar (Sony Research India), Pankaj Wasnik (Sony Research India)
• Date & Time: Saturday, December 16, 9:15 a.m. EST
• Location: Third Workshop on Efficient Natural Language and Speech Processing (ENLSP-III): Towards the Future of Large Language Models and their Emerging Descendants