The International Institute for Strategic Studies (IISS), a well-known research institution, recently released a report, pointing out that although the United States restricts the export of high-performance AI chips to China, these control measures may promote Chinese AI researchers to move towards “fields with lower computing requirements” and guide them to develop “new competitive advantages”.
The report states that in recent years, the capabilities of large language models have significantly improved, and OpenAI’s creation of GPT-3 in 2020 is an important milestone. These improvements are attributed to the creation of larger and more universal model architectures, the increase in dataset size, and the amount spent by technology companies to increase the computational power of training models. Empirical research has shown that there is a close relationship between the dataset size, computational cost, and parameter count of a given model, and in practice, computational cost is the strongest constraint for model improvement.
In addition, an increasing number of participants are developing large-scale language models and experiencing diffusion across multiple dimensions. Research on large-scale language models mainly occurs in the United States, but researchers from other countries, especially China, as well as specific research institutions in other places, such as DeepMind in the UK, have invested significant resources in building their own models. In addition, the types of institutions that develop language models have expanded to include both large technology companies such as Google and Microsoft, as well as dispersed research teams.
The report suggests that the proliferation of large model technology has two recent impacts on security: these models may produce higher quality and more content for false information, while competition for large model development may exacerbate geopolitical tensions.
The report also analyzed that in October last year, the US government announced new export controls on advanced semiconductor chips flowing to China, partly because these chips are crucial for the development of artificial intelligence. Although the established intention of export controls is to limit the development of artificial intelligence systems for surveillance or military applications, language models also highly rely on these advanced semiconductors. Part of the real motivation for implementing export controls may be to maintain the United States’ advantage in language modeling, whether as part of a broader competition strategy for artificial intelligence technology or because the government specifically wants to suppress China’s development in language modeling.
The report also doubts whether these measures are effective. Although large language models have been improved due to improved computing power, they cannot continue to do so at the current speed. Researchers are actively seeking to develop more computationally efficient methods to train similar models. One country attempting to limit the computing power of another country as a means of competing for the development of artificial intelligence may incentivize the target country to develop competitive advantages in these more computationally efficient artificial intelligence methods.
In addition, high-quality texts such as books and academic journal articles may soon become a more pressing limitation in the development of language models than the availability of computing power.
Countries may also increasingly view the development of large models as a sense of national pride, which may exacerbate competition for their development. The report is concerned that as large models become closer to the center of national technological competition, the government may be more proactive in cutting off access to language models developed by competing countries, further disrupting the internet.