AHEAD of OpenAI CEO Sam Altman’s four days in exile, several staff researchers wrote a letter to the board of directors warning of a powerful artificial intelligence discovery that they said could threaten humanity, two people familiar with the matter told Reuters.
The previously unreported letter and AI algorithm were key developments before the board's ouster of Altman, the poster child of generative AI, the two sources said. Prior to his triumphant return to OpenAI, more than 700 employees had threatened to quit and join backer Microsoft in solidarity with their fired leader.
The sources cited the letter as one factor among a longer list of grievances by the board leading to Altman's firing, among which were concerns over commercializing advances before understanding the consequences. Reuters was unable to review a copy of the letter. The staff who wrote the letter did not respond to requests for comment.
After being contacted by Reuters, OpenAI, which declined to comment, acknowledged in an internal message to staffers a project called Q* and a letter to the board before the weekend's events, one of the people said. An OpenAI spokesperson said that the message, sent by long-time executive Mira Murati, alerted staff to certain media stories without commenting on their accuracy.
Some at OpenAI believe Q* (pronounced Q-Star) could be a breakthrough in the startup's search for what's known as artificial general intelligence (AGI), one of the people told Reuters. OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks.
Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because the individual was not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said.
Reuters could not independently verify the capabilities of Q* claimed by the researchers.
'VEIL OF IGNORANCE'
Researchers consider math to be a frontier of generative AI development. Currently, generative AI is good at writing and language translation by statistically predicting the next word, and answers to the same question can vary widely. But conquering the ability to do math — where there is only one right answer — implies AI would have greater reasoning capabilities resembling human intelligence. This could be applied to novel scientific research, for instance, AI researchers believe.
Unlike a calculator that can solve a limited number of operations, AGI can generalize, learn and comprehend.
In their letter to the board, researchers flagged AI’s prowess and potential danger, the sources said without specifying the exact safety concerns noted in the letter. There has long been discussion among computer scientists about the danger posed by highly intelligent machines, for instance if they might decide that the destruction of humanity was in their interest.
Researchers have also flagged work by an "AI scientist" team, the existence of which multiple sources confirmed. The group, formed by combining earlier "Code Gen" and "Math Gen" teams, was exploring how to optimize existing AI models to improve their reasoning and eventually perform scientific work, one of the people said.
Altman led efforts to make ChatGPT one of the fastest growing software applications in history and drew investment - and computing resources - necessary from Microsoft to get closer to AGI.
In addition to announcing a slew of new tools in a demonstration this month, Altman last week teased at a summit of world leaders in San Francisco that he believed major advances were in sight.
"Four times now in the history of OpenAI, the most recent time was just in the last couple weeks, I've gotten to be in the room, when we sort of push the veil of ignorance back and the frontier of discovery forward, and getting to do that is the professional honor of a lifetime," he said at the Asia-Pacific Economic Cooperation summit.
A day later, the board fired Altman.
Reuters
Sun Nov 26 2023
Sam Altman led efforts to make ChatGPT one of the fastest growing software applications in history and drew investment - and computing resources - necessary from Microsoft to get closer to AGI. - REUTERS/Filepic
ISIS Malaysia's perspective of Budget 2025
An excellent rakyat-centric budget under the overarching principle of a caring and humane economy.
Budget 2025: Record increase in STR, SARA aid initiatives
The government will provide a significant boost to the Sumbangan Tunai Rahmah (STR) and Sumbangan Asas Rahmah (SARA) initiatives next year.
Budget 2025: EPF contributions to be made mandatory for foreign workers – PM Anwar
The government plans to make it compulsory for all non-citizen workers to contribute to the Employees Provident Fund (EPF).
What policies to expect from Indonesia's new President Prabowo
Prabowo will be open to foreign investment, his aide has said, such as by offering investors management of airports and sea ports.
Budget 2025: Govt allocates RM470 mil to empower women's participation in PMKS
The Women's Leadership Apprenticeship Program will be intensified as an effort to produce more female corporate personalities.
Israel sends more troops into north Gaza, deepens raid
Residents of Jabalia in northern Gaza said Israeli tanks had reached the heart of the camp, using heavy air and ground fire.
Indonesia ramps up security ahead of Prabowo's inauguration
Prabowo Subianto will be sworn in as Indonesia's president on Sunday with Vice President-elect, Gibran Rakabuming Raka, also taking office.
Immediate allocation of RM150 mil for local authorities, DID to tackle flash floods
Datuk Seri Anwar Ibrahim said this allocation is intended to address the recent flash floods that hit the capital and several major towns.
Budget 2025: Sabah, Sarawak to continue receiving among highest allocations - PM
Sabah and Sarawak continues to be prioritised under Budget 2025, with allocations of RM6.7 billion and RM5.9 billion respectively.
NFOF will be operational in November 2024 with funding of RM1 bil
PM Anwar Ibrahim said NFOF will support venture capital fund managers to invest in startup companies with RM300 million set aside for 2025.
Minimum wage to increase to RM1,700 effective Feb 1, 2025
The Progressive Wage Policy would be fully enforced next year with an allocation of RM200 million, benefiting 50,000 workers.
Bursa Malaysia ends higher on Budget 2025 optimism
The benchmark index, which opened 1.85 points higher at 1,643.29, moved between 1,641.71 and 1,649.31 throughout the trading session.
Five important aspects relating to people’s lives in Budget 2025 - PM
The focus is on driving the MADANI Economy, speeding reforms, cutting red tape, raising wages, and tackling the cost of living.
Economic outlook: Govt plans to leverage, expand existing city transit system
The expansion aims to provide a more efficient and reliable public transportation network, reduce congestion, and improve accessibility.
Economic outlook: Budget 2025 to lay foundation for a digital-driven economy
The report said Budget 2025 will entail efforts to position Kuala Lumpur as a top 20 global startup hub by 2030 through the KL20 initiative.
Economic outlook: Corruption and lack of accountability hinder economic progress
Special Cabinet Committee on National governance is established to curb corruption, law reforms to modernise outdate regulations, MoF said.
National Wages Consultative Council will be strengthened
The govt will also incentivise hiring women returning from career breaks, offer job matching and improve care services facilities.
Economic outlook: Ensuring 11 years of compulsory education for all children
Budget 2025 will continue prioritising upskilling and retraining initiatives to equip workers with the latest skill sets necessary.
Consolidated public sector projected to record lower surplus of RM41.7 bil 2024
The MoF said the consolidated general government revenue is estimated to increase slightly to RM384.7 billion in 2024.
PM announces substantial Budget 2025 hastening Malaysia to become Asian economic powerhouse
Datuk Seri Anwar Ibrahim said it would create jobs and also tackle financial leakages to enhance public spending efficiency.