AI News: Understanding Neurons, Benchmarking Chatbots, Unfaithful Explanations, and More
15th May 2023
Welcome to this week's round-up of AI research news.
Language Models Can Explain Neurons in Language Models
OpenAI has released a new paper titled "Language models can explain neurons in language models". The research uses automation to interpret the neuron activations in a large language model, applying a three-step process: explaining neuron activations with GPT-4, simulating these activations, and scoring the explanation. With this method, the team identified over 1,000 neurons with explanations scoring at least 0.8 in GPT-2 XL (suggesting that, according to GPT-4, they account for most of the neuron's top-activating behaviour).
Chatbot Arena: A Platform for Benchmarking LLMs
Chatbot Arena, a new benchmarking platform, provides randomized, anonymous battles for LLMs. At present GPT-4 (1st), Claude-v1 (2nd), and GPT-3.5-turbo (3rd) are leading the leaderboard, which is based on the Elo rating system. Keep an eye on this space as more commercial models like Bard are set to enter the arena soon.
Unfaithful Explanations in Language Models
new paper, "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting", uses a clever trick to demonstrate how a model's chain-of-thought explanations do not always reflect its reasoning process. In particular, by biasing the prompt fed to a model, the authors show the model's tendency to provide reasons for its decisions that don't reflect the basis for the decision itself.
Google's PaLM 2 Now Powers Many Products Including Bard
Google has released a technical report on PaLM 2, its latest model that now powers numerous Google products, including Bard. PaLM 2 incorporates compute-optimal scaling, improved dataset mixtures, and architectural and objective improvements. However, Sundar Pichai, at Google I/O, announced that PaLM 2 is not where they plan to stop - the team is already at work on Gemini, a model designed for multimodality, API integration efficiency, and future innovations like memory and planning.
DataComp: Exploring New Training Sets
DataComp is a participatory benchmark where researchers propose new training sets while the training code remains fixed. Participants can either propose a filtered subset from CommonCrawl or bring their own external data, which is then used to train a CLIP model with fixed hyperparameters. The authors conduct experiments ranging from 4 GPU hours up to 40,000 GPU hours on A100s and share insights about data scaling phenomena.
Meta AI's ImageBind: Aligning Six Modalities
Meta AI has released ImageBind, a joint embedding space that aligns six modalities, enabling cross-modal retrieval, arithmetic, detection, and generation. ImageBind pairs different data sources with images, including depth, heat maps, IMU, audio, and text. Models and code are released under a creative commons non-commercial license.
FrugalGPT: Reducing Cost and Improving Performance
The team behind FrugalGPT aims to reduce the cost of using large language models like GPT-4 while improving performance. They propose techniques like prompt adaptation, LLM approximation, and LLM cascade. In their experiments, FrugalGPT achieves a strong price-performance ratio relative to individual models.
InstructBLIP: Instruction Tuning for Vision-Language Models
"InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning" gathers a wide range of instruction-based datasets spanning different tasks to study instruction tuning in the vision-language model setting. This work also introduces an architecture that feeds instructions as a conditioning signal into the network at multiple points. Models and code have been released.
Detecting ChatGPT Imposters with a Single Question
In the paper "Bot or Human? Detecting ChatGPT Imposters with A Single Question", the authors explore the challenges modern Captcha systems face with text input. They explore tasks that are challenging for bots but manageable for humans, such as symbolic manipulation, noise filtering, randomness, and graphical understanding. Personally, I'm not looking forward to future captchas. An ever-increasing fraction of my time is spent finding squares containing traffic lights and motorbikes. And I do not like it.
CAMEL: Clinically Adapted Model Enhanced from LLaMA
CAMEL (Clinically Adapted Model Enhanced from LLaMA) is a LLaMA model that undergoes additional pretraining on clinical notes before fine-tuning on clinical instructions. Code and a demo are available, allowing you to inspect the outputs of the model.
Scaling Transformers
Vcc proposes to selectively compress input sequences to transformer layers, allowing transformers to scale up to 128 thousand tokens and beyond. The method preserves the most important, or "VIP", tokens, and aggressively compresses the rest before they are passed into a transformer layer. These are then decompressed to the original sequence length. Vcc remains fast at longer sequence lengths and uses less memory compared to baseline transformers and efficient self-attention schemes like Longformer.
Anthropic's Claude
On a similar note, Anthropic announced the expansion of Claude's context window from 9K to an impressive 100K tokens. This new development means that businesses can now submit hundreds of pages of material for Claude to digest and analyze. Read more
AI in the News
DeepMind co-founder Mustafa Suleyman has warned that AI will "create a serious number of losers", noting that "many of the tasks in white-collar land will look very different in the next five to 10 years". Read more
SnapChat Influencer Caryn Majorie has created an AI version of herself that can be your virtual girlfriend for a dollar a minute. Read more
IBM CEO has announced a hiring freeze, expecting AI to replace 7,800 jobs in the near future. Read more
China's AI industry appears unfazed by US export rules, suggesting that the country's AI development remains robust and resilient. Read more
AI Risk and Proposals
AI pioneer Yoshua Bengio shared a proposal for what he describes as AI scientists. These systems, which use Bayesian inference to answer questions, focus purely on truth in a probabilistic sense. They could help us understand diseases, develop therapies, understand climate changes, and find materials for efficient carbon capture. Bengio suggests a policy of banning autonomous AI systems that can act in the world unless proven safe. Read more
Reason Magazine recently hosted a discussion with Jaan Tallinn and Robin Hanson about whether we should pause the largest-scale experiments with AI. The conversation highlighted significant differences in opinion about the risks associated with training GPT-5 scale models and how the AI community should proceed. Watch here
Tool Roundup
In the world of AI tools, we have Otter, a multi-modal model with in-context instruction tuning, and Transformers Agent, an experimental API from Hugging Face that allows the user to define a set of tools and design an agent to interpret language to use these tools. Otter | Transformers Agent
Eric Hartford has released a model called WizardLM-7B-Uncensored, which is WizardLM trained with a subset of data where responses that contained alignment or moralising were removed. The intent is to provide a WizardLM that doesn't have alignment built in, so it can be added separately. Find out more
Resources and Challenges
For those eager to test their skills, Lakera has set up a challenge where you are tasked with prompting Gandalf to reveal a secret by bypassing safeguards. Try it out
The OpenAI cookbook is a valuable resource containing helpful snippets for working with the API. Check it out
A user named Quicksilver recently posted a sophisticated prompt design on the OpenAI discord that has garnered attention. Take a look
Learning Opportunities
LLM Bootcamp hosts a series of free lectures on YouTube covering various topics related to prompting and deploying large language models. They also discuss UX design for the LLM paradigm. Watch here
Cohere has launched a set of tutorials for large language models under the name LLM University. These tutorials cover how these models work and how they can be used to build apps. Learn more
AI Explained is a YouTube channel that provides regular updates on developments with AI and large language models. Subscribe here
The Inside View podcast offers a collection of interviews relating to AI safety and existential risk. Listen here
Prompt engineer Riley Goodside often posts insightful prompt-related threads on Twitter. A recent thread “Google Bard is a bit stubborn in its refusal to return clean JSON, but you can address this by threatening to take a human life” makes for interesting reading. Link to thread
Perspectives
Ke Fang describes the leverage that LLMs offers individuals, stating "With the support of GPT-4, I feel unstoppable... the overnight surge in productivity is intoxicating, not for making money or starting a business, but for the sheer joy of continuously creating ideas from my mind, which feels like happiness." Read more
An article by Rob Henderson titled "The Silent Strings of ChatGPT" has expressed concern about how ChatGPT may reinforce what he refers to as the emerging 'thought police'. It suggests that people may not rely on AI to learn the 'truth' regarding taboo or controversial topics, but rather, they may use these AI models to gauge what is permissible to say in polite society. Read more
Book Recommendation
This week, I recommend "Dealers of Lightning: Xerox Parc and the Dawn of the Computer Age" by Michael Hiltzik. This book offers an insightful history of a highly innovative R&D computing lab that developed foundational technologies like graphical user interfaces, laser printing, ethernet, the computer mouse, and many others. Find the book here
Filtir: a fact-checking API
Lastly, I'm working with colleagues on a project called Filtir. It's a fact-checking API specifically for AI-generated text. If you're interested in this problem, feel free to reach out to me.
If you prefer video summaries, you can find a video version of the newsletter here: