NYU Shanghai Awards First-ever PhD

Computer scientist Sean Welleck has become NYU Shanghai’s first-ever PhD graduate after successfully submitting and defending his dissertation several months ahead of schedule in December. 

Admitted in 2016 as part of the first group of candidates for NYU Shanghai’s PhD program in Computer Science, Welleck, a native of Austin, Texas, spent four and a half years in Shanghai and New York pursuing research in machine learning. Welleck graduates with nine publications at top conferences and will begin post-doctoral studies at the University of Washington. 

“Congratulations to Sean! NYU Shanghai is very proud of him and proud to celebrate this important milestone for our university,” said Chancellor Tong Shijun. “This is only the beginning for us as a comprehensive global research university - and I’m looking forward to celebrating the achievements of many more doctoral students in the years to come.”  

Another four students are expected to defend their dissertations later this year. NYU Shanghai, founded in 2012, currently supports 35 students pursuing doctoral studies in ten disciplines from data science to neural science to sociology. The university expects to offer even more advanced degree programs in the future. 

A High-Profile Researcher

Before joining NYU Shanghai’s PhD program, which is offered in collaboration with the NYU Courant Institute of Mathematical Sciences and the NYU Graduate School of Arts and Science, Welleck had worked for two years as a software engineer for IBM after graduating from the University of Pennsylvania with bachelor’s and master’s degrees in computer science. “The industry experience actually made me realize my dream was in research rather than in engineering, so I decided to do a PhD and encountered NYU’s Shanghai PhD program since I was interested in Professor Zhang Zheng’s research in deep learning,” Welleck said.

NYU Shanghai Professor of Computer Science Zhang Zheng, who is also director of Amazon Web Services Shanghai AI Lab, said Welleck was a well-rounded student. “In our first video call, he didn’t talk much but what he said was all on target,” Zhang recalled. “He has a good background in coding and we really liked the fact that he’s a marathoner, which showed he had the perseverance that a good researcher should have. We saw potential in him.”

Since the NYU Shanghai program enables students to build partnerships across the NYU Global Network, Welleck also worked closely with Kyunghyun Cho, Associate Professor of Computer Science and Data Science at NYU Courant. In 2017 and 2018, they published their findings on multiset prediction at the Conference on Neural Information Processing Systems (NeurIPS), one of the top international conferences in the field of machine learning and computational neuroscience. With both papers, Welleck, the first author, won the NVIDIA AI Labs Pioneering Research Award.

“Zhang has such wide experience in computer systems and machine learning, so he always provided me with creative perspectives that others who work in a single area their entire career might not have,” said Welleck. “During NeurIPS, he introduced me to many people he knew in the field to help me build my network in the machine learning community. For instance, Professor Marisa Carrasco, who I met at the conference, gave me very useful feedback from the perspective of neural science, which was eye-opening.”

After Zhang took on duties at Amazon in 2018, Welleck started to work more closely with Cho and researchers from Facebook’s AI Research Team (FAIR) where he was interning.

“Sean has published nine papers at top conferences in machine learning and natural language processing, accumulating more than 200 citations,” Cho said. “Whenever I think of Sean's successful PhD years, I recall early 2016 when I was asked by the admissions committee to interview Sean and feel how lucky we were to have admitted him and that he joined our inaugural PhD program at NYU Shanghai.” Zhang also said that Welleck’s achievements were beyond his expectations.

Teaching Machines to Write Sentences

After his summer internship was cancelled due to the pandemic, Welleck used his extra time to finish his dissertation and graduate a semester early. 

In his dissertation, Welleck focused on how machines generate text. He studied situations where machines failed or had odd behaviors when writing sentences. “For instance, in certain cases, the machines start repeating themselves. Or their language could be plain and lack fun,” Sean explained. “They may also logically contradict themselves. Imagine talking with a chatbot and it says ‘Hi, I’m a lawyer.’ Then when you ask about its job again, it says ‘I’m a doctor.’” While most training methods from the past told the machine learning model what it should do, Welleck adopted a new training method called Unlikelihood Training in which he also told the model what it shouldn’t do, which can increase the efficiency of training.

He also explored how to create natural language processing models to learn different orders in which to generate sentences, which is a new idea for the field called “non-monotonic generation.” “People write texts in certain orders. English and Chinese speakers write from left to right, but this order may not be ideal for machines. What if they write better from the middle or in some arbitrary order? We should explore these possibilities to keep improving text generation,” he said. 

A Marathoner and a Podcaster 

Beyond his research, Welleck found time to pursue hobbies as a marathon runner and podcaster. “I went running every morning. That was one thing that kept me grounded whether I was in Shanghai or New York. I really miss the time I ran along the Huangpu River and enjoyed the beautiful city view,” said Welleck, who also participated in several marathons in Japan during his years living in Shanghai.

Welleck also launched a podcast last June called The Thesis Review, conversations with machine learning researchers about their PhD theses, and how their research has evolved since.