GenAI: The Buy vs. Build Solution

As more organizations invest in artificial intelligence (AI) for competitive advantage, the build-versus-buy decision is becoming more critical. While buying access to an off-the-shelf solution such as ChatGPT or Copilot involves lower upfront costs, rapid deployment, and faster time to market, building a customized solution for your organization prioritizes tailored functionality, interoperability with your tech stack, and flexibility, control, and improved security.

With organizational spending on generative AI (GenAI) expected in increase by 76.4% this year over last year to $644 billion according to Gartner, computer scientists are in a position to understand the nuances in buy vs. build and advise their organizations, institutions, and start-ups on the most effective and efficient solution.

With GenAI, the distinct roles of computer scientist, data scientist, and large language model designer are converging, where computer scientists with experience in data and models are well-positioned to advise across the lifecycle of decision making, implementation, trouble shooting, and upgrading. “Computer scientists have a big role to play in the buy versus build decision because they can use their expertise to bring data, models, and software together,” said Anand Rao, Distinguished Services Professor of Applied Data Science and AI at Carnegie Mellon University’s Heinz College of Information Systems and Public Policy.

Buy vs. Build vs. Adapt

Buying an off-the-shelf GenAI solution involves obtaining a license from OpenAI, Perplexity, Google, or another provider for an already developed LLM platform such as ChatGPT, CoPilot, or Gemini. An organization can sign up users for the service on an enterprise or single-use license.

Building a GenAI model, on the other hand, involves leveraging one or more LLM to train a model on specific, required uses. Adapting an existing model to specific organization use cases is a step that more organizations are taking or considering. “When you blend something that you’ve built internally and host that on an existing AI model, you need to ensure that you focus and align that effort with your business objectives,” said Assaf Melochna, president and co-founder of Aquant, an agentic AI software company. “Lots of people say, ‘We need to do something with AI.’ but if you aren’t focused or aligned, you won’t solve your problems or get a good return on investment. You also need the ability to scale your solution and achieve tangible results.”

To sort through the decision, there are four main considerations: cost, time, data, and expertise.

Cost: Building involves the expense of using an LLM to generate responses, cloud storage for the immense amount of data generated by GenAI applications, and acquiring and compensating the talent needed to build a system. Gartner estimates that building a custom model from scratch includes $8 million to $20 million in up-front costs, and ongoing expenses of $11,000 to $21,000 per user, per year.

When buying, enterprise licenses are the major expense, but those can balloon quickly for a per-API (application programming interface) license. Buying enterprise AI licenses costs $100,000 to $200,000 up front, and $280 to $550 per user per year in ongoing costs, according to Gartner.

Adapting a model involves the cost of licensing, plus either contracting with a third party for assistance or hiring internally, plus cloud storage expenses. Costs for adapting vary widely depending on the use case. For example, creating a custom app through GenAI for content personalized to an organization is likely to cost $750,000 to $1 million in up-front costs, with ongoing costs per user per year of $1,300 to $11,000, according to Gartner.

Time: When building, the time to train a model may seem overwhelming, but only training will solve the problems that led to GenAI in the first place. The amount of time is dependent upon the use case. Specialized subject matter knowledge will likely take time to build. For a startup that sees time to market as critical, buying and adapting an existing LLM would make the most sense.

Time will also depend upon the robustness of your internal talent and whether they can manage initial and ongoing tasks related to AI business needs.

Data: Building becomes a necessity for an organization that depends on specialized knowledge or proprietary information. Buying is an option for businesses that depend on generalized knowledge from an off-the-shelf GenAI platform. Layering proprietary data on top of an existing model is a use case for adapting.

Expertise: In technical fields such as law, healthcare, science, or other areas where technical knowledge is paramount, building is an imperative because buying isn’t likely to get what’s needed. If special expertise isn’t needed, an off-the-shelf solution in an option, as is adapting.

While GenAI can provide the expertise that many organizations seek, GenAI doesn’t yet have the capacity to supply the experience—or the judgment—of subject matter experts that an organization may want to tap, Rao said. “If you have that unique subject matter experience that professionals with decades of experience have that you want to tap and codify in some way, you’ll want to build, because you need that expertise reflected in your AI solution,” he said.

Many organizations are going with built solutions because the salaries of AI engineers who can build a custom solution are “through the roof,” according to Erik Linstead, a professor of artificial intelligence and machine learning at Orange, CA-based Chapman University. High-end AI researchers are negotiating $250-million pay packages amid a talent war between Facebook, Google, Open AI, and other tech firms. AI engineers—including prompt engineers, AI researchers, and MLOps engineers—are receiving offers that include salaries between $300,000 and $400,000, and sometimes 2% to 5% equity packages.

“For the most common use cases I’m seeing, which is generally knowledge discovery and interrogation within a historical knowledge base at a company, a buy solution will usually work,” Linstead said. “Many companies want to make it easier for rank and file workers to find the historical knowledge that already exists in the company.”

Mistakes to Avoid

In the rush to embrace AI, leaders could make ill-considered decisions that come back to haunt their organizations. The ongoing commitment to maintaining an AI investment, and updating that investment as necessary, requires careful consideration. “Do you have a fully fledged IT team infrastructure?” asked Sunil Dua, director of client success at 108 Ideaspace, a business management consultancy in Toronto, Canada. “Do you have the resources to maintain what you have?” Expecting an IT team that likely already has its hands full to master and maintain an entirely new function is unrealistic and likely to lead to disappointment and could risk the return on an AI investment, he said.

Underestimating the compliance, back-office, privacy, and cybersecurity issues involved in AI is a common mistake, and can be costly. For example, 78% of Chief Information Security Officers polled believe that AI-powered cybersecurity threats are having a significant impact on their companies, according to Darktrace.

Those unclear on what they want from AI should consider starting with smaller steps in lieu of a larger investment, suggested Linstead. “The first step may involve ensuring your staff has familiarity with GenAI and understands how to create prompts to get the information they need,” he said. “Or, using customized tools for workflows that increase efficiency. Obviously, there are risks of moving too slowly, but there are also risks involved in investment that may not match your needs.”

Amy Buttell is a Silver Spring, MD-based technology, legal, and business journalist, content creator, writer, and ghostwriter.