January 2023

This month's newsletter covers yet more rapid developments in generative AI, concerns over misuse of facial recognition technology, and a roundup of academic research following NeurIPS'22

Personal updates

My method for generating counterfactual explanations for regression models was published in a recent U.S. patent filing.

Business

Another month of generative AI news. Some notable software engineering assistants for generative AI hit the market recently, like Bito AI for explaining code snippets, automatic test creation from GitHub Copilot Labs, semantic code search with Buildt, automatic commit message generation, and pull request generation using Tensai. DoNotPay continues to use deepfake technology to simulate customers’ voices to call center agents. GPTZero, a student project to detect plagiarism in essays written by ChatGPT, also garnered interest. Parth has published an intriguing investigation into reverse-engineering Copilot. At the same time, concerns over the quality of generated output are growing. New evidence has emerged showing that AI code generators can reproduce copyrighted code nearly verbatim, and users of AI code generators are more likely to produce insecure code. StackOverflow has introduced new policy that bans the posting of solutions made with ChatGPT as being too unreliable.

Facial recognition in the news. In Louisiana, facial recognition technology used by the Jefferson Parish Sheriff’s Office resulted in a false arrest of a Black man on 2022-11-25. In New York, Madison Square Garden Entertainment used facial recognition technology to ban a lawyer from attending a Rockettes show with her daughter, simply because her law firm was involved with litigation against MSG Entertainment. The same technology was used to ban another lawyer from attending a Knicks game, for similar reasons. Yet another lawyer barred from a Mariah Carey concert is now suing over MSG’s blanket ban for all attorneys whose firms are involved in lawsuits against them.

Meta. On 2022-12-01, Real Women in Trucking, a nonprofit advocacy for women truck drivers, sued Meta alleging sex and age discrimination in showing job ads on Facebook. This suit follows Meta’s settlement with the Justice Department on 2022-06-16 requiring an overhaul of Facebook’s ad delivery system to prevent the use of protected characteristics. On 2023-01-04, Meta was fined €390m by Ireland’s Data Protection Commission, constituting once of the largest fines for GDPR violations to date.

Leaked footage from iRobot Roomba vacuums were recirculated on Scale AI and other data annotation platforms.

Tencent’s 异次元的我 (Different Dimension Me; a service on the QQ app), a Lensa-like service for turning photos into anime-like cartoon, was found to turn a dark-skinned girl into a cartoon ape.

DoNotPay’s AI legal assistant, which claims to advise a defendant on what to say in court, is set to debut in a court case next month.

Policy and government

New York City’s Department of Consumer and Worker Protection has delayed enforcement of Local Law 144 of 2021 (on Automated Employment Decision Tools) until 2023-04-15. A second public hearing will be held on 2023-01-23 ahead of an anticipated final rulemaking of 2023-02-15.

☛ Submit comments here by 2023-01-23.

[FR] On 2023-01-31 at 10am EST, the U.S. Equal Employment Opportunity Commission (EEOC) will hold a public hearing on Navigating Employment Discrimination in AI and Automated Systems: A New Civil Rights Frontier.

☛ Sign up for the webinar on Zoom.

[The Register] The Cyberspace Administration of China has announced new regulations around “deep synthesis services”, requiring explicit watermarking of generated content, data privacy controls, and prohibiting illegal uses or those deemed against national interest. These regulations come into effect on 2023-01-10.

Papers

[arXiv] Jingling Zhang et al. Why do people judge humans differently from machines? The role of agency and experience. Presents moderate statistical evidence that human decisions are judged more on intent than harms created, whereas the opposite is true for machine decisions.

[doi] Margaret A. Webb and June P. Tangney, Too Good to Be True: Bots and Bad Data From Mechanical Turk, Perspectives on Psychological Science. An engaging narrative of quality filtering of survey responses, resulting in just under 3% of crowdsourced responses considered to be valid.

[arXiv, Twitter] Daniel Paleka and Amartya Sanyal, A law of adversarial risk, interpolation, and label noise. Proves a tight lower bound on misclassification errors due to adversarial labels on general classifiers. The need for an exponentially large number of data points to safeguard against interpolation of adversarial data shows that inductive structure of classifiers must be considered in order to explain real world adversarial attacks.

[arXiv] Yaren Bilge Kaya and Kayse Lee Maass, Leveraging priority thresholds to improve equitable housing access for unhoused-at-risk youth. Uses queuing models to simulate the allocation of shelter beds to the homeless, especially high risk youths which are disproportinately likely to abandon queues for homeless resources, whose results suggest that having more beds will lead to a large drop in queue abandonment.

[Twitter] Yuntao Bai et al., Constitutional AI: Harmlessness from AI Feedback. Anthropic AI show that self-critique and self-revision of responses by conversation AI can reduce the likelihood of generating harmful responses and improve the likelihood of responding meaningfully to provocative prompts.

[arXiv] Florian Tramèr et al., Considerations for differentially private learning with large-scale public pretraining. Argues that data leakage between public and private data sets contaminate current research seeking to augment privacy-preserving ML with public data, leading to overstated results that would diminish when distributional shifts grow in real world settings.

[arXiv] Stephanie C. Y. Chan et al., Data distributional properties drive emergent in-context learning in transformers, NeurIPS’22. Shows that learning in transformer models is influenced not just by architecture, but also on the approximate power-law distribution of data; empirically, they see a trade-off between in-weight and in-context learnings.

[arXiv, Twitter] Gowthami Somepalli et al., Diffusion art or digital forgery? Investigating data replication in diffusion models. Shows that Stable Diffusion reproduces training data about 2% of the time.

[doi] Yujia Li et al., Competition-level code generation with AlphaCode, Science. DeepMind’s reinforcement learning-trained code generation model performs about as well as the median participant in programming competitions, although the authors note some brittleness over the specific wording of problems.

[arXiv] Avanika Narayan et al., Can foundation models wrangle your data? Shows that GPT-3 can achieve state of the art results in data management tasks such as entity matching, error detection, and imputation, even without fine-tuning.

[arXiv] Neil Perry et al., Do users write more insecure code with AI assistants? Shows that university students are likely to accept the output of AI code generation tools verbatim without further testing and were much more likely to submit insecure answers to computer security programming tasks. The authors note that many users also experimented with prompt engineering and query repair when the answers returned were unexpected.

[arXiv, GitHub] Pan Lu et al., A survey of deep learning for mathematical reasoning. Reviews the various tasks that have attracted machine learning solutions and developments in benchmarking and data availability.

[arXiv] Seyed Mehran Kazemi et al., LAMBADA: Backward chaining for automated reasoning in natural language. Uses backward reasoning and task decomposition to query language models, creating chains of reasoning that outperform current state-of-the-art like chain of thought (CoT).

[OpenReview, Mastodon, GitHub, video] Ali Shahin Shamsabadi et al., Washing The Unwashable : On The (Im)possibility of Fairwashing Detection, NeurIPS’22. Proposes a method to analyze interpretable surrogate models for fairwashing risk, i.e., being prepared in a way to be less discriminatory than the original model.

[arXiv] Tilman Räuker et al., Toward transparent AI: a survey on interpreting the inner structures of deep neural networks. Provides a taxonomy for understanding interpretability at different levels of resolution.

Articles

Deborah Bloom, Born of eugenics, can standardized testing escape its past?, Undark. Reviews the dark history of standardized testing.

Reports

QuantumBlack by McKinsey, The state of AI in 2022—and a half decade in review. McKinsey’s latest report shows strong adoption of AI on unstructured data for service operations, while technical roles like data scientists remain difficult to hire for.[Twitter]

Center for Countering Digital Hate, Deadly by design: TikTok pushes harmful content promoting eating disorders and self-harm into users' feeds. In-depth analysis of content shown to teen accounts shows that TikTok frequently recommends content on body image, mental health and eating disorders. Separately, Arvind Narayanan argues that TikTok’s success is not due to the recommender system per se, but rather its design and data availability.

Information Commissioner’s Office, Tech Horizons Report, December 2022. Highlights data protection concerns in four key emerging technologies: distributed finance, immersive technology (e.g., augmented reality), internet of things, and consumer healthtech.

IQT Labs (In-Q-Tel, Inc.), AI Assurance Audit of RoBERTa, an Open source, Pretrained Large Language Model. Runs several experiments for bias and system-level reviews for other issues like software dependency reliability and security.

Software

[GitHub, Twitter] Eluther AI release Pythia, a suite of tools and language models for interpreting autoregressive transformers like the most recent generation of LLMs.

[arXiv, GitHub] Wenzhuo Yang et al., OmniXAI: A library for explainable AI. Provides a toolkit of explainable AI techniques.

Data

[arXiv, GitHub] Sérgio Jesus et al., Turning the tables: biased, imbalanced, dynamic tabular datasets for ML evaluation, NeurIPS’22. Provides a synthetic data set for bank account opening fraud, containing customer age as a protected class.