What CEOs Need to Know About AI

What is AI?

Today, when people discuss artificial intelligence, they often refer to a class of AI tools called large language models (LLMs). These are tools like ChatGPT, Claude, and Gemini.

However, under the umbrella of artificial intelligence, there are many other classes of models, like Computer Vision (CV). This guide focuses on the language-model assistants and agents built on LLMs (Large Language Models) — one important subset of generative AI, which also spans image, audio, and video generation. LLMs are the class of AI popularized by OpenAI's ChatGPT product.

Executive Summary: What CEOs Need to Know about AI

• The same prompt can produce different results each time it is run
• AI can make mistakes
• AI has a limited working memory; strange things can happen when it's full
• Just like human employees, AI can perform much better when it has the right tools and information available to do its job

How do LLMs like ChatGPT and Claude work?

The most common chatbot experiences let an LLM collaborate with you to create a document (i.e., a chat history). You type in a prompt, and the LLM creates a response. That response is built from the LLM's unit of work: the token. Think of tokens as stand-ins for words, word fragments, punctuation, and even special symbols that signify the end of a response.

Each token is generated by considering the entire document up to that point, and then selecting from a collection of likely next tokens. Then the model considers the document again, including the new token, to generate the next one.

Imagine taking turns with a friend, building a castle from Lego bricks. Your friend starts building the foundation, and then you start building the walls. Then your friend takes a turn, looking at what was built before to place the next bricks, one after another.

A large language model works similarly: it looks at the cumulative text and, based on what it sees, it adds to the conversation until it reaches a natural stopping point (a special end-of-response token). Instead of using bricks, you're building with tokens. The foundation of the conversation is the system prompt — high-priority instructions supplied by the provider or the app you're using, usually not shown to you — and on top of that, any additional context you gave to the model. This process of building the response one token at a time based on all of the previous tokens is the magic of AI.

Limitations

These AI systems have a few curious limitations. First, LLMs aren't guaranteed to see your entire chat history. LLMs have a limited working memory, or context window. Any information that falls outside the context window can't be considered by the AI. If your Lego castle gets too big, the AI system can only see the most recent parts of it.

Context sizes vary widely by model and product: as of mid-2026, many top models sit around 200,000 tokens, and a few reach a million or more. In English, each token averages a bit less than one word (this varies by language, code, and formatting). So if you have a long conversation with an LLM, at some point it won't "remember" the first part of your conversation—even if you still see it in the chat history. (Products handle this overflow differently—some drop the oldest text, others summarize it.)

Working around context windows

Part of the tooling built around the most advanced large language models allows the model to do things like search through text outside the model. Think of it as the search functionality in your word processor. There are also tools like compression that take a large amount of text and summarize it into a smaller token size, like an abridged version of that text.

Another common approach is to use agents. In other words, the AI you're talking to spawns off multiple additional AIs and delegates work to them to help divide up a task that otherwise would be difficult or impossible for a single AI to do itself.

What kind of mistakes can LLMs make?

Hallucinations are probably the most common kind of LLM mistake. An AI hallucination is a confabulation; it claims something that isn't true.

For instance, I have seen LLMs add new, incorrect information to a document summary. I have seen them get general facts incorrect.

Related to hallucinations, LLMs are unreliable calculators. They can reason well about math, but generating an exact answer token by token is not the same as computing it, so long or precise calculations often come out wrong. Many AI products integrate calculators and software environments into their harnesses so that the LLMs can get reliable calculations without relying on the model itself to do the arithmetic.

LLMs can also omit or ignore important information. For instance, it isn't uncommon to see an LLM ignore a direct instruction if it is part of a complicated set of instructions.

Assumptions: LLMs can make poor assumptions about the purpose or desired results of a prompt.

Herd mentality: Sometimes AI can take cues from your own prompts that inform its behavior. It can be important to either use a new context to ask important questions so they aren't contaminated by previous discussion, or to ask for an independent review of an item you want an opinion on.

LLMs also tend to write like LLMs—formulaic phrasing and predictable structure that a careful reader learns to spot. Editing AI drafts back into your own voice is part of the work.

What are AI agents?

In 2026, the word "agent" is loosely used, much like "the cloud" and "blockchain" were overused when cloud computing and cryptocurrency were trendy. People love to say they have agents for travel, email, investing, and anything else they can think of.

The word agent usually means an AI system that pursues a goal on its own—taking an action, checking the result, and deciding the next step, often using tools along the way.

In some contexts, it might point to an autonomous business function. If it sounds like someone is trying to impress you with their use of agent, this is most likely what they mean. You might have agents that periodically research marketing opportunities in social media, or ones that pull analytics data and make SEO suggestions.

In a more technical context, the word agent or sub-agent can indicate when one AI process delegates tasks to other AI processes to complete work in parallel. For instance, to write an essay, Claude might spin off an agent to search the web for references, another agent to outline the topics, and a final agent to synthesize the information and write the essay content.

The important information conveyed by the word agent is autonomy: once launched, an agent can run many steps on its own. The best-designed ones still pause to checkpoint for human approval before consequential or irreversible actions.

What tasks should you delegate to AI first?

Because of the power held by the CEO role, you should start applying AI to tasks well within your area of expertise before branching out. The reason is simple: AI can make mistakes and misunderstand your prompts. By starting well inside your area of comfort, you will become familiar with both the kinds of mistakes AI can make and how to best detect and prevent these mistakes.

One easy place to start is to have the AI research your business and provide a strategic overview of what your company does and how it fits within the competitive landscape. Not only will the results fall entirely into your area of expertise, but AI's outsider perspective will also likely produce insights that you find both surprising and actionable.

Research Acme Inc using its primary website, the media, social media, competitors, and any other relevant resources. Create a strategic overview of the business, including the competitive landscape, its competitive strengths and weaknesses, the ideal customer profile, and biggest risks and assets. Include direct quotes (with links) to supporting documentation.

I expect that you will disagree with some aspects of the AI's analysis, which is perfectly normal. Sometimes this is an AI mistake, but often this indicates an opportunity for your marketing and website content.

Beyond research, ask AI to critique your writing or strategy. Again, don't take the results at face value, but as you would a consultant's advice. You know much more about the specifics of your business than the AI. Again, part of the exercise is understanding the limits of AI and how to provide better context and clearer prompts to get the results you want.

As you become more comfortable using AI, you can use it in more autonomous applications, such as helping to prioritize your work and monitor your business.

How to get better results with AI

Context and tooling are the two keys to improving the performance of AI systems. Context helps the AI understand your situation, your role, what you value, what level of risk you're comfortable with, what writing style you prefer, what data is available, and what your other preferences are. Tooling helps an AI system retrieve data and perform actions more reliably.

In Claude Code, you can provide context for a directory (folder) with a CLAUDE.md file. In Codex, that is AGENTS.md. In Claude Cowork, you can provide project and folder instructions that give greater context. In all of these mechanisms, the context is automatically incorporated with your prompts, and it can refer to other documents which the AI can then use as necessary to produce better results.

Both OpenAI and Anthropic also support Skills, which can provide both context and tooling to their AI systems. Because skills provide both instructions to the AI and software it can run, they greatly expand what AI can get done—both on your computer and out in the external world.

Reviewing AI work: looks done vs. is done

Just like with a new employee, AI may not get all the subtext from a set of instructions you give it. Standards for completing work vary between organizations, and definitions definitely vary for AI models. If you want something done in a certain way, you must be specific, or the context you provide must provide those specific answers. If writing a certain kind of document in your organization requires a certain format with a table of contents, page numbers, and so on, you must be explicit in that or again provide the proper context in a way that the AI will automatically discover and apply it.

This is easier said than done, however. The more complex a project is, the more likely it is that you will not provide adequate instructions for AI to know when the job is done. And even if you do, there is still a chance that AI will forget a step, take a shortcut, or make an assumption you would not have made.

Now I recommend that people consider, before they work on a project, how important it is that that project is complete and correct. There are two types of AI projects. Projects that improve your luck surface area and are less critical in terms of quality and correctness, and projects that are key work products. If you are sending out cold sales emails, it's inevitable that some of them will not work. For that application, it may be okay if AI-generated output isn't 100% correct or successful. You may not care if all the recipients are not fully qualified leads.

On the other hand, a marketing page on your website probably needs to be correct and done to a high standard. In these cases, it's important to include quality criteria in your prompt along with feedback loops that ensure that that standard of quality is achieved. In software, this is more straightforward because you can ask specifically that automated tests be written to confirm the software works. On something like a marketing page, it's harder—there's no test suite to run. In those cases, an independent review pass at the end helps: a second AI, ideally from a different provider so its blind spots don't match the first one's, sweeps the output for problems. (I built one such tool, Impasse, but the principle matters more than the product.)

Work at the beginning of a project can also help result in a much better output. Asking most modern AIs to plan a project will often produce much better results than a project that you do not ask for a detailed plan. The detailed plan helps keep the AI on track, and it also gives you an opportunity to review the work before it is executed and make sure that what it plans actually matches your desired output. On complicated, critical projects, time spent in the planning phase pays off—up to a point. Planning has diminishing returns and doesn't replace testing and review, but a solid plan gets the result much closer to done by the time the AI says it's done.

In any case, however, for critical work, it is important to have human review of the output. As good as the most advanced models are, they can still make mistakes. For a deeper treatment of how to direct AI so the output is actually done and not just plausible, see directing AI to ship real software.

Where does your data go?

Every business has sensitive information. A basic SaaS business has sales numbers, vendors, agreements, and perhaps HR documentation that is both confidential and sensitive. A medical device company needs to worry about patient confidentiality, perhaps intellectual property such as trade secrets, and clinical data. It is therefore very important for a business to understand what information is put into AI systems and what those AI systems do with that data. Many free AI tools use what you type—including files and other snippets of context—to help train future models. That is not the same as publishing your data or letting Google index it; your prompt doesn't become publicly searchable. But it does mean your confidential material has left your control and been absorbed into a vendor's training process, where you can't retrieve it. It's not a good idea to give sensitive information to a free AI tool. And the risk isn't limited to free tools: some paid plans also train on your data by default. Read the terms rather than assuming that paying buys you privacy. (More on the specific exposure of pasting internal processes into AI tools: AI tools and trade secrets and where AI quietly goes wrong.)

Making poor decisions about how AI uses your data can result in issues such as breaches of confidentiality and loss of intellectual property.

This is one reason why on-device AI models are becoming increasingly interesting for businesses. If you run the AI model on your own hardware, the chance of having that data misused is much lower. For example, I am dictating this article right now using an on-device dictation model instead of one that sends my data to a cloud AI provider. The most powerful on-device models require significant hardware resources to run.

Most businesses still choose foundation model providers like Anthropic or OpenAI for their most complicated AI work. Their business and enterprise plans generally offer stronger contractual confidentiality—typically no training on your data, with defined retention. Whether that is adequate depends on your data sensitivity and any regulatory obligations, so review the terms against your own requirements rather than assuming the default fits.

Outside of very narrow use cases, free AI tiers are generally a poor choice for business data. Businesses should also be cautious about AI features bundled into other software products—these are sometimes used as a conduit for collecting private information to train more advanced models. When a tool you already use quietly adds an AI feature, check what its terms now say about training on your data.

Does your company need an AI policy?

If your business has employees or contractors with access to confidential or sensitive information, such as source code, revenue numbers, customer lists, your business plans, etc., you need an AI policy. While it has long been a business model to use private data in order to optimize advertising targeting you, it is a completely new business model to use private business data to train AI systems your competitors can use.

On top of the privacy concerns, there are also concerns that AI can provide your employees with false or misleading data and low-quality work products. Depending on your location, there also may be new or emerging legislation that restricts how you can use AI in business.

The practical reality is that shadow AI is already happening inside your company whether or not you've addressed it—a written policy is what lets the cautious half of your team use AI safely instead of quietly. A good policy names approved tools, classifies which data may and may not go into them, sets least-privilege limits on what tools an agent can touch, and requires human approval for consequential actions. If you don't have one yet, the free AI Policy Generator will draft a starting point in a few minutes.

What does AI cost?

Depending on what you're doing, AI can seem either extremely cheap or extremely expensive. There are two basic ways to pay for AI systems. The first way to pay is by the API call; both OpenAI and Anthropic charge per million input tokens and per million output tokens. The cost varies quite a bit by model. As of mid-2026, input runs from a fraction of a dollar per million tokens for small models up to roughly $10–$15 per million for premium ones, and output from a few dollars per million up to $75 or more. These figures move constantly—treat them as rough illustration and check the provider's current pricing page before budgeting.

The other way to pay for AI is through packaged plans, which typically run from about $20 per user per month to $200 per user per month. For predictable, interactive use, a packaged plan usually costs far less than paying for the same volume through the API—though the plan's rate limits and fair-use rules cap how much you can actually consume.

If these measures seem abstract and confusing on the API side, I agree. It is not rare to see stories of companies accidentally overspending on their AI budget, especially when they are incentivizing their employees to use AI.

Most small- to medium-sized businesses would be wise to start with a packaged plan, like Claude Team. Packaged plans keep spending predictable and give you a chance to learn the cost-benefit ratio of AI before committing to usage-based API billing. For how the tiers compare, see Claude Team vs. Pro vs. Enterprise.

The model landscape in one page

As of mid-2026, the major frontier providers are Anthropic, OpenAI, and Google (with its Gemini models). Many operators reach for Anthropic or OpenAI first, but which provider leads shifts by task, price, and release—so evaluate them on your actual workload rather than on a fixed ranking.

These providers offer their services through packaged plans and APIs. Models from Anthropic and OpenAI are also available through third-party cloud providers like Amazon and Microsoft.

In addition to these frontier providers, which offer proprietary models, there are open-weight models like Llama, Kimi, DeepSeek, and Qwen. (Open-weight isn't the same as open-source—the weights are published, but the license still governs commercial use, so check each model's terms.) Depending on your computing hardware, these models can be run locally. This can reduce the model costs, while increasing your capex, maintenance, and IT costs. Running a model locally, depending on your agent runtime, can also increase your privacy and security. These open-weight models are generally a step behind the latest frontier models but capable for many purposes, and some perform better than others depending on the exact application.

It is also possible to run these open-weight models through a paid cloud service as well.

Finally, there are small models that are cheaper and require fewer resources to operate. Many of the high-end frontier models have small variants for less intensive use. The small models are adequate for some routine tasks, and are sometimes configured to switch to more powerful models when a task seems more complex.

Do you need to train your own model?

In general, training, fine-tuning, or distilling a model should be your last resort. If you're having difficulty reliably applying AI to a task, you should attempt better prompting, better examples, and the use of techniques like RAG first.

As a rule of thumb, fine-tuning is best for very narrow applications, or for teaching a model a consistent style or behavior it won't reliably follow from instructions alone. If your goal is simply a strict output format, that is usually better handled with structured-output features and validation than with fine-tuning.

Where to start as the CEO

Use AI yourself, daily, on real work before you mandate anything for your team. You can't direct work you've never done, and the fastest way to learn where AI helps and where it fails is to feel both firsthand.

Once you're ready to roll it out more broadly, pick your first workflows deliberately rather than by enthusiasm. Weigh three things for any candidate task:

• Value — will doing this faster or more often actually move a business objective?
• Cost of error — what happens if the output is wrong and slips through? A cold-email draft and a regulatory filing sit at opposite ends of this scale.
• Data sensitivity — does the task require feeding in confidential or regulated information?

The best first projects are high-value, low-cost-of-error, and low-sensitivity. Start there, give each one an accountable human owner, define what "done" and "reviewed" mean, and measure the result before expanding.

A practical first-90-days sequence:

1. Inventory the shadow use. Assume your team is already using AI; find out which tools and for what, without blame.
2. Write the one-page policy. Approved tools, what data may and may not go in, and who to ask when unsure. The AI Policy Generator will draft it.
3. Pick two low-risk, measurable workflows and run them with a named owner and a defined review step.
4. Measure, then expand to higher-value or higher-stakes work only once the review process has proven it catches mistakes.

Glossary of AI terms

Token: A symbolic representation of a word, word part, punctuation, symbol, or number. A tokenizer converts text into tokens and back; tokens are the units an LLM reads and generates.

Context Window: The "working memory" of an AI model—the maximum number of tokens it can consider at once, shared across the system prompt, your conversation, any retrieved material, and the model's own output. When a conversation exceeds the window, the product has to drop or summarize the oldest content.

Agent: An AI system that pursues a goal in a loop—taking actions, checking results, and choosing the next step, often with tools and human approval checkpoints.

RAG: Retrieval-Augmented Generation. RAG is the technique of looking up or searching for information to inform an AI system's response. For instance, when asked how to replace a car's battery, the AI system might search for the car's manual and then search within that manual for information on how to replace the battery.

Fine-tuning: The process of shaping a trained AI model's output by continuing training on a specific behavior.

Inference: The use of an AI model to generate an output from an input.

System Prompt: High-priority instructions, supplied by the provider or the application you're using, that shape the model's behavior. They usually sit at the start of the context and are typically not shown to the end user.

MCP: Model Context Protocol. An open standard for connecting an AI application to external tools and data sources. MCP standardizes the connection; each connection still needs its own authentication and permissions.