The big GPT-3 Wave

Quite a lot of hype and discussion between experts, peers and friends recently on the shaping of a future GTP-3 market.

GPT-3 is a natural language processing (NLP) model currently being adopted by chatbots and other AI assistants.

As part of the OpenAI group, GPT-3 generates AI-written text from sentences, paragraphs, articles, short stories, dialogue, lyrics, and more.

But don't confuse GPT with Generative AI.

In essence, GPT is a statistical model that predicts word sequences. A pre-trained language model that can be applied to various tasks without labelled data (or very few) and achieve nontrivial results.

Giant datasets (from places like Common Crawl, Wikipedia, and more) are typically used to train such a model, which adds to its power.

Another model called Bidirectional Encoder Representations (BERT) was created by Google for NLP. The BERT algorithm helps Google understand the context behind user searches instead of just matching keywords. It's a significant step forward in the vector-based search domain. Hence, why we see other players like you.com making moves.

Increasing market share

The market has shown significant growth for GTP-3-based products. Many applications and use cases have seen market investment over the last five years.

Recent advances in computing infrastructure have enabled chatbots to dominate much of the GPT-3 space due to their ability to provide a satisfactory customer experience - despite some dissatisfaction with overall "prompt engineering" capabilities for system-2 tasks requiring multi-step reasoning.

In other words, chatbots are smoke and mirrors for most complex use cases. The chatbot is an NLP interface with behind-the-curtain defined rules and recognition behaviours tailored to their domain. However, you may be scratching your head for a while if you want complex reasoning from Chatbots using thousands of parameters. In other words, finding chatbot technology which can scale reasonable intelligence when faced with perplexing or complex input.

A crossroads for investors

Nonetheless, GTP-3 appears to be attracting much interest from big players. And it's pretty exciting.

Others have started branching out into other GTP-3 sub-spaces, such as copywriting (see Jasper ai & Ryter ai) and GAI text-to-image conversion (see Mid journey and DALL-E 2).

In contrast, for the open source world and other research players as well as think tanks, GPT-3 is used to decipher specific problems, such as operational efficiency and automation.

So, the staggering advancements and results in the GTP-3 community are pretty hard to ignore.

Already, we are seeing medium-sized players outside of the AI space cherry-pick certain architectural elements of GTP-3 to model and train on their own data. Take Lawra.ai for example. Expect this trend to pick up momentum over the coming year.

A battle between open source and big tech

Even more interesting, though, we have a compelling market competition evolving before our eyes.

We can see specific research labs responsible for training and maintaining datasets now expanding their services into the industry through open source.

Open source provides an easy access point for individuals or groups to download such datasets and models to experiment with.

However, it requires significant hardware resources to crunch and execute.

But what is a problem is an opportunity.

Will we see more cooperatives like BigScience ship minor, less hardware-intensive GTP-3 models?

I think so. We will undoubtedly see new platforms or communities follow the same idea, allowing open source communities, research labs, and think tanks to share models and data.

Now, that is a bit of a punch in the face for big tech. 🧐

At the same time, we can see many prominent tech players now trying to figure out how best to accommodate the advancement of GPT-3 within their own platforms.

So, I guess the million dollar question is - to build on top of GPT-3 and focus on "vanilla" NLP products or spend $100 million on compute to train your own proprietary data model to eventually open source and solve real-world problems?

Such a question will either make or break the GPT market. Either it will push forward competition and therefore market growth or potentially cause indirect market fragmentation.