Why the only way to ride the company AI wave is experimentation – Big Think

Sometimes I think back to what life was like BC, before ChatGPT. Things that seemed impossible then, things we thought of as science fiction, things many were sure couldnt be done. A large number of those things are now reality.

We finally have a machine that can see, hear, and which we can actively talk to about anything for hours on end. It can teach us and code for us and write for us and quiz us. It can paint and draw and create 3D art or even entire games.

Naturally, the market responded. We are increasingly finding ways to make use of this, and to make ourselves more efficient. Among the first jobs under threat were copy editing and graphic design. Theres also talk of generative AI applying to paralegal and research jobs, and theres even discussion of how it will affect sales jobs inside Google.

What weve made is the first fuzzy processor, where outputs arent rigidly predictable based on the inputs, unlike everything weve known in software so far. It can take any data in any format, and transform it into any other. Text to image. Image to movie. Movie to text. Speech to code. Code to diagram. Code to code in a different language. You name it, its now possible. And this changes almost everything we think we know about what the corporate world should look like.

Large language models (LLMs) fall somewhere between a probabilistic piece of software and a brand new employee who doesnt know anything about your company. Which means pretty much the only way to test whether theyre useful is to try them out. To experiment. There are plenty of evaluations and tests and leaderboards, but none of them are good enough to tell you who to hire. The contours of its abilities are still unknown, and in most cases these contours will only be known after you start using them. Researchers call this a jagged frontier.

Its like the cloud or mobile or the internet: All the possibilities in the future are inchoate, and will require redesigning what you have. Three-year transformations arent going to help. The way to make this work within organizations is hard, because by definition this doesnt easily fall into someones bucket of responsibilities. You might not get immediate ROI, but the costs are real.

Try BigThink+ for your business

Engaging content on the skills that matter, taught by world-class experts.

You cant just entrust a few architects with the new way of doing things. Instead the only way to generate value from AI is to have people across the organization in all siloes and in all domains actively engage with it. A Harvard study alongside Boston Consulting Group, found that using AI improves the work for their consultants rather substantially, across task types.

Consultants using AI finished 12.2% more tasks on average, completed tasks 25.1% more quickly, and produced 40% higher quality results than those without. Those are some very big impacts.

AI seems to even out skills across the organization, upskilling the least skilled folks and leveling the playing field. And despite the benefits the study also found areas where using AI made the output worse: where recipients were less equipped to ask the right questions or use the advice that AI gave. This, again, is the jagged frontier. Its really difficult to know a priori the best way to use AI.

If you think about the fact that LLMs are an anything-to-anything converter, then the biggest benefit is that it allows you to retool everything. It allows you to do whatever youre doing, much better, faster and cheaper. You can have drastically smaller teams producing output equivalent to those made by 10 times the employees elsewhere. Its an enormous Archimedean lever! Once it works, that is. If it works.

Every organization has an enormous number of software systems it uses to run daily. These systems are kept in sync mostly by fragile humans who interpret the output of one to be the input for another. This will shift, and for it to work you need to see a new way of doing things.

Dont just think about processes which need to be mapped or schemas that need to be designed, but instead about how you can train or use an AI to do things for you. To think as an employee would. It will mean a large chunk of resources need to be redirected.

Large language models fall somewhere between a probabilistic piece of software and a brand new employee who doesnt know anything about your company.

So how might you actually try to do this? One way to get started is, well, to start with the easiest parts, for example: writing text and reports, especially internal ones; creating marketing materials or slide design; writing code to glue things from one system to another. You could test out GPT-4 and Claude 2 for some of these, and see which parts they seem to succeed at, followed by, for instance, Mistral so you know how to use open source models and combine them to get the actual pipeline you want. You could try Stable Diffusion for images.

Were so early its still extremely unclear if it makes sense to think of the use cases like the cloud, or like another API provider, or like a processor. And the answer might well differ for each use case.

Contrary to almost all software wisdom weve learnt over the past two decades, with LLMs its much easier to go more general first in capabilities, rather than more specific. You cant start in a niche if youre training your own model: there might not be enough data, and the model might not learn enough logic. Its much better to do broad pre-training and then fine-tune to your context. Its easier to teach logic and reasoning first, like sending it to university, rather than memorization of new domain-specific facts. This is true whether youre talking about microlithography or additive manufacturing or target protein identification.

In fact the true surprise of Gen AI is that it took creating an AI which can do almost everything before it got good enough for the few things it is getting used for.

So the way to make use of AI is finding ways of stacking it to solve your particular problem. This means you need to try for more audacious programs without clear applications and tinker.

The best way to play the situation is to keep your options open. If youre building things, use open source as much as possible: They keep getting significantly better, and theyre almost as easy to use as the existing incumbents. Big Techs adoption of the technology has happened at warp speed, and they will be winners here regardless of what happens, but when making decisions on what to build for your use case, theyre not trying very hard to be flexible.

This means the question is not about finding ever more specific use cases and then identifying the easiest way to solve them, but rather committing to creating something that could answer many of the questions you have, and then trying to coax them to get better at answering questions.

The reason this is hard is because its the exact opposite of how we deal with software. You cant get precise requirements or user feedback first before knowing what to build. You cant create unit tests that are specific or easily measurable first. Going narrow first is a surefire way to fail. The only way is one of exploration.

We shouldnt just plug AI, LLMs or otherwise, into our current systems and hope it will work; we should use it to build better ones. But theres good news. This is unlikely to be an area where there are a couple of clear winners you can seamlessly deploy in a year or two. Its likely to remain a place where you win based on the combination of technologies you bring in, the way you mix them together, and the way you make them work within the existing organizational construct.

Subscribe for a weekly email

Which means the fact that you know your industry or process or niche better than anyone else will remain a competitive advantage, assuming you use it to build something brilliant. Youll never get a perfect solution that can solve your problems and increase EBITDA with no effort, no matter what the software vendors promise, so you need to build those muscles yourself, just like with cloud or with mobile or with the internet.

We shouldnt just plug AI, LLMs or otherwise, into our current systems and hope it will work; we should use it to build better ones.

AI has a duality. Its both a tool and a catalyst. It takes over routine tasks, but only by helping reimagine those tasks as they exist today. It democratizes a large amount of expertise and helps juniors level up, but has hard edges that arent obvious when they dont work. Recognizing this is the first step to a restructuring of what the business of business might look like.

The key question is not which roles it can do or what jobs are gone forever, it is whats the price of expertise being made free?

See the original post:

Why the only way to ride the company AI wave is experimentation - Big Think