LANGUAGE models have become a key factor when it comes to creating the most thorough and accurate artificial intelligence possible. The new model developed by Microsoft and Nvidia is said to feature about 530 billion parameters and to be capable of exceptional accuracy, especially in reading comprehension and complex sentence formation.
Nvidia and Microsoft's Megatron-Turing Natural Language Generation model (MT-NLG) marks a new record for a language model. According to the tech firms, their model is the most powerful to date.
Thanks to its 530 billion parameters, it is able to outperform OpenAI's GPT-3 as well as Google's BRET. Specialized in natural language, it is able to understand texts, reason and make deductions to form a complete and precise sentence.
Language models are built around a statistical approach. While many methods exist, it is the n-gram model that is being used here.
The learning phase enables analysis of a large quantity of texts to estimate the probabilities that a word will 'fit' correctly in a sentence.
The probability of a word sequence is the product of the probabilities of the words previously used. By using probabilities, we can create perfectly grammatical sentences.
Biased algorithms still an issue
With 530 billion parameters, the MT-NLP model is particularly sophisticated. In the field of machine learning, parameters are often defined as the unit of measurement for machine performance.
t has been repeatedly shown that models with a large number of parameters ultimately perform better, resulting in more accurate, nuanced language due to their large dataset.
These models are capable of summarizing books and texts and even writing poems.
To train MT-NLG, Microsoft and Nvidia created their own dataset of about 270 billion "tokens" from English-language websites.
In natural language, "tokens" are used to break up text into smaller chunks to better distribute information.
The websites included academic sources such as Arxiv, Pubmed, educational websites such as Wikipedia or Github as well as news articles and even messages on social networks.
As always with language models, the main problem with widespread, public use is bias in the algorithms.
The data used to train machine learning algorithms contain human stereotypes embedded in the texts.
Gender, racial, physical and religious biases are widely present in these models. And it is particularly difficult to remove these problems.
For Microsoft and Nvidia, this is one of the main challenges with such a model. Both companies say that the use of MT-NLG "must ensure that proper measures are put in place to mitigate and minimize potential harm to users."
Before fully benefiting from these revolutionary models, this issue needs to be tackled, and for the moment it seems far from resolved.
ETX Studio
Wed Oct 13 2021
Language patterns reach record highs, but questions remain. - ETX Studio
Iran's supreme leader says Hamas leader's death will not halt 'Axis of Resistance'
The "Axis of Resistance", built up with years of Iranian support, includes Hamas, the Lebanese Hezbollah group, the Houthi movement in Yemen, and various Shi'ite groups in Iraq and Syria.
Putin says Russia willing to seek compromises between Iran and Israel
Russia is ready to help seek compromises between arch-foes Israel and Iran, President Vladimir Putin said on Friday, saying these would be difficult but possible.
What proposals will Russia push at the BRICS summit?
The proposal is also to establish a BRICS reinsurance company to allow uninterrupted shipment of goods and key commodities between members.
Indonesia's free meals plan in the spotlight as Prabowo readies for office
Prabowo calls the programme one of the main drivers of economic growth, eventually set to add an estimated 2.5 million jobs.
Astro AWANI's revamped English news website, AWANI International, launches on Oct 21
Astro AWANI's revamped English platform delivers in-depth global news and expert analysis to keep you informed on key developments.
Israeli strikes kill 33 people in Jabalia refugee camp in Gaza, medics say
Residents of Jabalia said Israeli tanks had reached the heart of the camp after pushing through suburbs and residential districts.
Liam Payne's ex-partner calls for media restraint after 'painful' death
Cheryl Tweedy used her statement to urge the media to remember they had a seven-year-old son, Bear, who could read the reports.
Analysts: Indonesia's strong MoF leadership team to boost investor confidence
Sri Mulyani Indrawati as head of Indonesia's Ministry of Finance is expected to instil confidence among investors.
Biden offers both a carrot and a stick to Israel as his term nears an end
Israel has frequently resisted US advice and has caused political difficulties for the Biden administration.
Putin says BRICS will generate most of global economic growth
Russian President Vladimir Putin will host a summit of the group in the city of Kazan on Oct. 22-24.
ISIS Malaysia's perspective of Budget 2025
An excellent rakyat-centric budget under the overarching principle of a caring and humane economy.
Budget 2025: Record increase in STR, SARA aid initiatives
The government will provide a significant boost to the Sumbangan Tunai Rahmah (STR) and Sumbangan Asas Rahmah (SARA) initiatives next year.
Budget 2025: EPF contributions to be made mandatory for foreign workers – PM Anwar
The government plans to make it compulsory for all non-citizen workers to contribute to the Employees Provident Fund (EPF).
What policies to expect from Indonesia's new President Prabowo
Prabowo will be open to foreign investment, his aide has said, such as by offering investors management of airports and sea ports.
Budget 2025: Govt allocates RM470 mil to empower women's participation in PMKS
The Women's Leadership Apprenticeship Program will be intensified as an effort to produce more female corporate personalities.
Israel sends more troops into north Gaza, deepens raid
Residents of Jabalia in northern Gaza said Israeli tanks had reached the heart of the camp, using heavy air and ground fire.
Indonesia ramps up security ahead of Prabowo's inauguration
Prabowo Subianto will be sworn in as Indonesia's president on Sunday with Vice President-elect, Gibran Rakabuming Raka, also taking office.
Immediate allocation of RM150 mil for local authorities, DID to tackle flash floods
Datuk Seri Anwar Ibrahim said this allocation is intended to address the recent flash floods that hit the capital and several major towns.
Budget 2025: Sabah, Sarawak to continue receiving among highest allocations - PM
Sabah and Sarawak continues to be prioritised under Budget 2025, with allocations of RM6.7 billion and RM5.9 billion respectively.
NFOF will be operational in November 2024 with funding of RM1 bil
PM Anwar Ibrahim said NFOF will support venture capital fund managers to invest in startup companies with RM300 million set aside for 2025.