Global InformationPaLM information
 | |
| Developer(s) | Google AI |
|---|---|
| Predecessor | LaMDA |
| Successor | Gemini |
| Available in | English |
| Type | Large language model |
| Website | ai |
PaLM (Pathways Language Model) is a 540 billion parameter transformer-based large language model developed by Google AI.[1] Researchers also trained smaller versions of PaLM, 8 and 62 billion parameter models, to test the effects of model scale.[2]
PaLM is capable of a wide range of tasks, including commonsense reasoning, arithmetic reasoning, joke explanation, code generation, and translation.[2][3][4][5] When combined with chain-of-thought prompting, PaLM achieved significantly better performance on datasets requiring reasoning of multiple steps, such as word problems and logic-based questions.[1][2]
The model was first announced in April 2022 and remained private until March 2023, when Google launched an API for PaLM and several other technologies.[6] The API was initially available to a limited number of developers who joined a waitlist before it was released to the public.[7]
Google and DeepMind developed a version of PaLM 540B called Med-PaLM that is fine-tuned on medical data and outperforms previous models on medical question answering benchmarks.[8][9] Med-PaLM was the first to obtain a passing score on U.S. medical licensing questions, and in addition to answering both multiple choice and open-ended questions accurately, it also provides reasoning and is able to evaluate its own responses.[10]
Google also extended PaLM using a vision transformer to create PaLM-E, a state-of-the-art vision-language model that can be used for robotic manipulation.[11][12] The model can perform tasks in robotics competitively without the need for retraining or fine-tuning.[13]
In May 2023, Google announced PaLM 2 at the annual Google I/O keynote.[14] PaLM 2 is reported to be a 340 billion parameter model trained on 3.6 trillion tokens.[15]
In June 2023, Google announced AudioPaLM for speech-to-speech translation, which uses the PaLM-2 architecture and initialization.[16]
- ^ a b Narang, Sharan; Chowdhery, Aakanksha. "Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance". ai.googleblog.com. Retrieved 17 March 2023.
- ^ a b c Chowdhery, Aakanksha; Narang, Sharan; Devlin, Jacob; et al. (2022). "PaLM: Scaling Language Modeling with Pathways". arXiv:2204.02311 [cs.CL].
- ^ Cite error: The named reference
venturebeatwas invoked but never defined (see the help page). - ^ Bastian, Matthias (5 April 2022). "Google PaLM: Giant language AI can explain jokes". the decoder. Retrieved 17 March 2023.
- ^ "Google: Why Is No One Talking About PaLM". seekingalpha.com. 12 December 2022. Retrieved 17 March 2023.
- ^ Vincent, James (14 March 2023). "Google opens up its AI language model PaLM to challenge OpenAI and GPT-3". The Verge. Retrieved 17 March 2023.
- ^ Huffman, Scott; Woodward, Josh. "PaLM API & MakerSuite: an approachable way to start prototyping and building generative AI applications". Retrieved 17 March 2023.
- ^ Singhal, Karan; Azizi, Shekoofeh; Tu, Tao; et al. (2022). "Large Language Models Encode Clinical Knowledge". arXiv:2212.13138 [cs.CL].
- ^ "MedPaLM: New Chatbots Will Soon Be Better Than Waiting For A Doctor". The Medical Futurist. 17 January 2023. Retrieved 17 March 2023.
- ^ Matias, Yossi; Corrado, Greg (14 March 2023). "Our latest health AI research updates". Google. Retrieved 17 March 2023.
- ^ Driess, Danny; Xia, Fei; Sajjadi, Mehdi S. M.; et al. (2023). "PaLM-E: An Embodied Multimodal Language Model". arXiv:2303.03378 [cs.LG].
- ^ Driess, Danny; Florence, Pete. "PaLM-E: An embodied multimodal language model". ai.googleblog.com. Retrieved 17 March 2023.
- ^ Edwards, Benj (7 March 2023). "Google's PaLM-E is a generalist robot brain that takes commands". Ars Technica. Retrieved 17 March 2023.
- ^ Lardinois, Frederic (May 10, 2023). "Google launches PaLM 2, its next-gen large language model". TechCrunch. Archived from the original on May 10, 2023. Retrieved May 10, 2023.
- ^ Elias, Jennifer (16 May 2023). "Google's newest A.I. model uses nearly five times more text data for training than its predecessor". CNBC. Retrieved 18 May 2023.
- ^ "AudioPaLM". google-research.github.io. Retrieved 2023-06-30.