学进去

‘Small Data’ Are Also Crucial for Machine Learning

Many people relate “artificial intelligence” with “big data.” There’s a reason for that: some of the most prominent AI breakthroughs in the past decade have relied on enormous data sets. Image _________ made great progress in the 2010s thanks to the development of ImageNet, a data set containing millions of images hand sorted into thousands of categories. More recently, GPT-3, a language model, was trained on _________ online texts to produce humanlike text in Jan, 2021. So it is not surprising to see AI being tightly connected with “big data” in the _________ imagination. But AI is not only about large data sets, and research in “small data” approaches has grown extensively over the past decade. The so-called transfer learning serves as an especially _________ example.

Also known as “fine-tuning,” transfer learning is helpful in settings where you have _________ data on the task of interest but abundant data on a related problem. You need to first train a model using a big data set and then retrain slightly using a smaller one related to your _________ problem. A research team working on German-language speech recognition, _________ , showed that they could improve their results by starting with an English-language speech model trained on a larger data set. Then, they used transfer learning to _________ that model for a smaller data set of German-language audio.

Small data approaches such as transfer learning are more _________ than more data-intensive methods. They can promote progress in areas where little or no data exist, such as in forecasting natural hazards that occur relatively __________ . In this context, small data approaches will become increasingly important as more organizations look to diversify AI application areas and invest in previously __________ fields.

Despite the progress in research, transfer learning has received relatively little __________ . While many machine learning experts are likely familiar with it at this point, the existence of techniques such as transfer learning does not seem to have reached the awareness of the broader space of policymakers in positions of making important decisions about AI funding and __________.

As long as the success of small data technique like transfer learning is __________ , resources can be allocated to support their widespread use. In that case, we can help correct the popular __________ regarding the role of data in AI and foster innovation in new directions.