Name: How Command A is Changing the Game for Enterprise AI Adoption
Uploaded: 2025-04-29T15:46:10.073Z
Duration: 28 min 44 s
Description: How Command A is Changing the Game for Enterprise AI Adoption

Transcript for "How Command A is Changing the Game for Enterprise AI Adoption": we are ready to start. So hi, everyone. Thank you so much for joining us today for this special Command a webinar. Over the next half an hour, we're going to take a deep dive into the most exciting generative model release we've yet to have, our newest model, Command a. CommandAid came out about a month ago now and represents our strongest generative model that we have developed to date. Today, you'll hear directly from myself and two of my colleagues, Nick and David, as we dive into what are the most capabilities and the unique value it brings to the table from the performance to the efficiency as well as security. And before I dive in, I want to emphasize we'd love to hear your questions, thoughts, and comments. If you have any, and we really hope you do, please post them in the chat. Our Cohere technical experts are here to provide detailed answers and ensure that you leave with a clear understanding of command a and its potential applications. But let's kick us off with a quick round of intros. So hi. Myra is Jesse, and I'm Cohere lead engineer and program manager for post training of all of our command models, which makes me responsible for execution across model training, evaluation, data, product operations, all the way to go to market. For Command a, my role was to manage the end to end program across the company to ensure the success of Command a from ideation and development all the way through to the launch in March. Next up, Nick, could you introduce yourself quickly? Yes. Hi, everybody. My name is Nick Jackobi, and I am a director of product for the modeling org. Cohere. And what does that mean? That means that, for command a, my team of product managers own kind of all of the capabilities of the model. So, we're working on safety and multilingual and instruction following and coding and all the different types of behaviors that command a exhibits. And, our job is to understand what our customers want from the models and then help work with the modeling team and the and the machine learning engineers to make sure our our our models, do what our customers need them to do. And with that, I will pass to David. Thanks, both. I'm David. I lead solution architecture at Cohere. Solution architecture at Cohere is specifically responsible for working with our customers to make sure they're successful with our models and products. So I work very closely with both Jesse and Nick to make sure that, one, feedback is making its way back into our products so that we're continues to improve the models. And two, that the model that we do ship and develop, we're able to get those in the customer's hands, make sure they get them deployed, and that they're successful with their prompting and implementation of the products themselves. Excited to be here, and thank you all for joining. Perfect. Fantastic. Thanks a minute, David. Alright. So to kick us off, let's talk a bit about performance. This is the cornerstone of what your users will truly care about and where the bar for expectations is set truly high. As I alluded to earlier, Command a is the most performant model that we have created to date. Our design philosophy has emphasized the importance of creating a model that excels particularly at enterprise tasks, agentic use cases, and across 23 different high priority languages for global enterprises. Let's look at the next slide while we'll be able to glance into the actual raw performance numbers that command a has. There's a few things when you look at this slide that I really want to call out and bring your attention to. Firstly, let's start off by discussing the performance of general enterprise tasks. Broadly speaking, it's quite hard to find an accurate proxy among publicly available generative model evaluation benchmarks for something that truly captures what enterprises care about. Most benchmarks that you hear about tend to measure a very specific type of behavior or capability, such as general knowledge, ability to reason, or performing simple programming tasks that you might expect in software engineering interviews, for example. While these benchmarks are able to shed light into the very specific scenarios, we recognize how complex and unique enterprise environments are. And throughout our development process, we wanted to make sure our model is able to flexibly perform in a variety of different enterprise settings. Therefore, we have created a bunch of internal evaluation sets, which are intended to mimic the types of prompts and scenarios that we have learned about through our extensive experience of working with enterprise of all shapes and sizes. As David already mentioned before, this is one of his cups of tea of gathering and understanding really what enterprise settings look like and what they care about. To guarantee fairness for these internal evaluation sets, they're done in a purely blind setting. Human evaluation experts are asked to choose the response to a prompt that they prefer from two models, and they cannot see whether a response came from our model or another model that we are comparing against. We've also provided thorough training to the human evaluators to ensure their preference aligns with what enterprises really care about, such as behaving like a chatbot when appropriate while only providing the answer in other instances while maintaining a professional and brand aligned tone throughout. As you can see from the bar charts on the very left of your screen, we are competitive with some of the strongest and oftentimes significantly larger models that exist out there. However, this is not to say we don't care about publicly available benchmarks. In particular, there are exceptions that help us evaluate the specific model capability, such as agentic behaviors in a realistic setting for an enterprise customer. A prime example of this would be TowBench that probably a bunch of you have already heard about, which you can see in the middle of the benchmark section on the right. The reason I want to call out TowBench is that it provides two different settings, one with a retail company, the other an airline, where the model is asked to behave like a customer support chatbot in accordance with company's internal policy, and it will be evaluated on its ability to successfully follow the company's policies in various different scenarios. This benchmark is fantastic as it not only provides a realistic enterprise setting, but it measures the model's capability in agentic tasks, which refers to the model's ability to autonomously behave and make decisions as opposed to following explicit orders from the user in order to successfully execute on a task. In this case, it would mean successfully responding to a user's inquiry. When it comes to agent evaluations, including TopBench, Commande is able to match or beat competitors consistently, as you can see from the charts on the right hand side of the screen, showcasing strong agentic behavior and performance. Finally, I cannot not talk about performance without mentioning more bilingual. This is a vital part of our models and the way that we want to actually have our models behave. It doesn't matter how good your model is if it can't support business in the language that your customers need. More so, it's not just enough to fluently speak the language. You also need to ensure the model can converse into correct dialect, pick up on cultural nuances, and complete a variety of complex tasks in the same way models can in English. Overall, we've done a lot of work to figure out the 23, which we believe to be the highest priority languages for businesses worldwide, and we're consistently showing strong performance that allows businesses to engage with the model in the language that they and their users operate in. If you look at the middle of the screen, you will see Command a showing strong performance against one of the strongest models out there when we actually put enterprise use cases again in front of human evaluators. All of this comes with native support for retrieval of vendor generation and tool use. In order to provide the optimal experience, we have built the model to support a context window of 256,000 tokens to ensure the model can perform well on all types of enterprise documents. But when you go up in context length, models often start requiring a lot more hardware to run and the experiments blow up at this point. This is something we closely kept in mind to make sure that while the model is as powerful as possible, it is also efficient and you are capable of running it at scale. I just wanted to to, again, emphasize how, interesting the the human evaluations are on the left hand side of this. And the reason is that these are these are blind evaluations of humans, of model response on, a wide variety, of prompts. And these are actually internally, these are the evaluations that we look at the most because they're they're they're unsheatable. The public benchmarks on the right where we're comparing to other people's public public numbers are gameable, and and it is possible for competitors and and anybody to kind of overfit their models to get, particularly high scores on these bank funds. I'm not saying that anybody is doing that necessarily, but we know that the numbers that we can trust are those, human evaluation numbers on on the left. And so as you can see from those, we have roughly the same performance as, two very popular phone tier models, GPT-four o and DeepSeq. And so the reason why it's important to make that point that we have the same performance as GPT four o and DeepSeq is when it comes to efficiency, we are much better than those models. So on the right, you can see a graph of throughput, and you can see that, we are nearly double the, throughput of GBD four o and nearly three times the throughput of, DeepSeq. This is partly just because of, the footprint of these models, so just the hardware that it runs on. GPG four o, we don't know what hard hardware it runs on. It's a closed model, operated on OpenAI servers. We do we don't we don't actually know how big it is. We can only just measure the speed of the output. But DeepSeek is an open weights, open source model, and Command a is an open weights model. Command a is a 11,000,000,000 parameters, and that 11,000,000,000 parameters model runs on two GPUs. So, you know, two of the kind of better standard GPUs, you can get from NVIDIA like h one hundreds. Command a will run on two h one hundreds. DeepSeek is over 600,000,000,000 parameters, so nearly six times the size of, command day in terms of parameters, and runs on, 16 h one hundreds rather than two h one hundreds. So that means if you want to run a copy of, DeepSeek privately deployed on your own internal hardware, you need to buy six 16 GPUs, whereas for, command a, you just buy, two. And command a will still be operating nearly three times as fast. So that means that, if you need to hit a certain throughput, maybe you have a big workload, and a certain QPS that you need to achieve, you're actually gonna have to provision deep seek even more in order to be able to, catch up with, command a. So to achieve the same throughput as command a, you might have to you might have to have enough, GPUs to do three copies of deep deep seek, whereas you only need one copy of command a to serve a certain, QPS. So you can see that although on the slide before, we see comparative performance in terms of quality, the the really transformational thing about command a is its size and its efficiency, in comparison to, DeepSeek and GPT four o and and leading contenders of, similar quality. And, just to give you an idea of we we worked very, very hard on the efficiency of Komodo, and there's all sorts of technical tricks and, clever things as we did under the hood to make it as fast as it is now. And just to show you, we we've got a demo here, we're just about to show you of, Command a compared to the previous our previous best model, which is Command r plus. And this is on a prompt, quite a to generate a long document, and you can see command a on the left. And our previous model, command r plus, which is actually a slightly smaller model, on the right, is just much slower. This is running on exactly the same hardware. So that gives you an idea of the of the speed ups and the efficiency gains our very clever Inference team managed to achieve, working on, command a. And, with that, I will hand over to David. We can jump into the security and how folks can actually use these models. So while we talked about performance both on the actual quality of the model and on the speed at which we can generate outputs, this only matters if you're able to use these models. Cohere operates under the philosophy that that we wanna make sure these models are all available where your data resides and not require companies to bring the data to our models. What this means in practice for those who may not familiar with all the various deployment methods in which you can use large language models today, Cohere is available across the spectrum of deployment options, meaning we have our own cloud hosted service where you can access our models via API endpoint, and we don't have to or you don't have to actually use or control any of the serving of the model itself. We're also available on almost all the major cloud providers and their various services, and we do private deployment where we can actually ship the model directly to you, and you can use and deploy that model inside of an environment of your choice. That could be hardware you've procured or leased from a cloud provider. That could be your own private cloud or even on premise in your basement running on your GPUs. The idea behind this flexible options when it comes to deployment is that everyone has various systems for how they want to run these models. So security becomes very important, especially as we're starting to send more and more crucial data to these systems. As RAG and agentic use cases are becoming the most commonly used, use case for these models in the market. Both of those involve sending a bunch of proprietary data to the models. RAG has to tap into your internal knowledge stores. And agentic systems, while also tapping into knowledge directly, are also increasingly moving into the action space, meaning they might actually be calling APIs or systems inside your company. While some of these use cases might be deemed non sensitive and then you're comfortable running via a cloud provider, some of these likely carry very sensitive workloads and traffic inside of them. So running those inside of your environment is the safest way to ensure that data is not leaking back into, another model's training or potentially being exposed to risks that you don't control, being a a company's hosting security in itself. The other side that's often talked less about with security is the model itself. So while Coherence wants to ensure that companies can use these models in whatever system they're set up in, we also want to ensure that companies feel comfortable using the models in production. What this means is that because Cohere is a model training company, we train our models from scratch, making sure that we have a full audit trail of every data source that went into that model. We can then work with companies to make sure they have comfort levels around the data the model was exposed to and the behaviors that it can expect to emerge from that model. And we also provide guarantees around ensuring the quality of outputs and making sure our customers are protected against any suits around IP and, risk, for example. Making sure that we can deploy the models and that the models themselves are safe to use, we believe are two fundamental pillars for making sure enterprise AI can actually be adopted inside of a company. When it comes to deploying the model themselves though, often the idea of private deployment is linked solely to security, meaning the model's running inside of your four walls. Therefore, the key benefit here is that you're able to, you know, run sensitive data inside that model. The other less talked about benefit with running models on premise is the control and customization you get. So whereas using a cloud hosted service such as Cohere's API does add a level of convenience and that you don't have to manage the internal infrastructure or hardware, you can start to run into issues when your production workload might require some custom configuration to that setup. Let's say you have a very high volume of traffic going through these models. And while Cohere might optimize our serving footprint to meet the general needs of the populace, which might optimize towards latency, you might need to optimize more towards throughput to make sure that you're actually able to meet the traffic demand without having to scale up that number of models too dramatically. By running a model instance in your environment, you have full control over being able to adjust the serving settings, how many GPUs you wanna put that model on, and really be able to tweak the actual serving pipeline and how you're routing requests into that. Additionally, if you're a company that operates on various cloud providers, you're able to now move that model wherever you get GPU capacity. So if you're working off of one cloud provider and you don't have enough h one hundreds there, you have complete freedom to move that model into another environment, which might not be possible if you're using various hosted applications. To Nick's point as well, we make sure this is very efficient to run. So comparable open source models, we see a very, very strong performance when using us on prem even if you're using other heavily quantized models from Hugging Face. I think these pillars make sure that command a is not just performance as Jesse and Nick already talked through, but that it's practical to use in production and really move on to scaled workloads. Cohere believes that last year, a lot of people were just getting their feet wet with models and really moving through POCs and pilots and testing these applications. But as companies are moving more towards production heavy workloads and volume is going to increase dramatically, the ability to run these models in an environment actually have very flexible hardware configurations and be very efficiently run on top of those various hardwares will become more and more important, especially as volume is going to only increase. I see there's a bunch of questions coming in, so we should have a couple minutes here at the end to address any of those. But other than that, thank you all for joining. We'll also be sharing some contact information, and feel free to reach out over LinkedIn or reach out to Cohere directly if you wanted to learn more or wanna have a conversation about any of your use cases. Thank you so much, David and Nick. We've definitely gotten a bunch of questions, so let me try to just make sure that I, prioritize the most common ones that we've seen. Starting with, Nick, I think this one might be good for us all, but they would love to hear your thoughts as well. We've seen some questions around real life use cases. There's something particular we've noticed where command a seems to outperform competitors well or where we've seen a lot of success in these types of cases. Would you have any examples to share with crowd? Yes. I do have a great example, actually. So this is my favorite example of Command a working really, really well is, a company called Qinet, q I n e c t. And they, build, learning software which helps people negotiate, and it's actually a company run by a whole lot of professors at at, INSEAD MBA university. And so it's, it teaches negotiation, and it's quite a complicated use case because it's a chatbot which has a whole set of rules and incredibly complex business case scenarios that it's working for, helping people role play both sides of, of different situations and has to keep track of various kind of emotional states and states of negotiation and very complex rule playing, role playing, rule following, decision tree making. And, they were using g p t four o up until now, and informed me very recently that they tried command a, and, we're noticing no difference in performance. So it's still doing a very good job, and it was just running twice as fast. And that just made all the difference in the world in terms of user experience. So I don't know how indicative of that is. I mean, I think it's obviously very addictive, but, it's a lovely example of us just working really, really well as all the metrics, shall be worth. Love this example. David, do you have any, stars on yours or anything to share about, successes we've seen with customers? Sure. What I would, share on the capability side, we're working with a customer on natural language to SQL type use case where they wanted to be able to more effectively query their documents. While we were getting some decent results with command r plus, the day we launched command a, we switched them over to that model, and we saw a massive increase in their quantitative assessment on the quality of SQL that was being generated and the ability for the model to actually correct itself and operate inside of a a looping agent style work flow, which was previously not the target area for command r. So command r was generating decent SQL in single shot examples, but this multi turn capability was a bit limited compared to the command a. We saw a massive improvement there, so the quality uplift was felt immediately while still staying on that same hardware footprint and actually being more efficient to run compared to Command r that they were using before. Amazing. Love those examples. Thanks, David and Nick. One question we've got in quite a bit is availability on cloud platforms, outside of Cohere. So at the moment, we are available on Azure, if you go take a look, as well as OCI. For AWS, we unfortunately do not have a confirmed date, but we will definitely let people know publicly, if and once there is one in place. So if you need any further details, we'll be happy to also follow-up, on the exact ones where they are available in AWS or or, sorry, where they're available, mister and OCI. OCI. We've also gotten a bunch of questions that I've seen here around North and its role, in this. So Nick, as also the product lead for North, maybe you could help disentangle a small bit around, what is the relationship of command to North at Cohere. We have our generative models, our large language models, and then we also offer models for embed and, embeddings and re ranking search results. And, our customers use our models to to build various things, And one of the things they've often tried to do in the past is to build retrieval augmented generation, systems that do things like q and a over, over company documents and company, email and messaging systems, and general kind of productivity type applications to help their workforce and the company be as good as, good as it can be. The more technical of our customers are able to build them this themselves. But what we found, over the past, I would say, eighteen months or so is that it is a lot harder than it looks. And so we decided to build our own application and our own platform, which combines all of the models together and is privately deployable, behind your firewall. And then, you have all of the models, and there are connectors there to connect to all of your company data. So, you know, all fully, authenticated, behind your firewall. So to connect to your your email, your Slack, your docs, your, your SharePoint, what whatever your company runs on. And then there is a, a a user in interface, which on the surface looks a bit like ChatGPT, but it's much more powerful than ChatGPT, and is able to to call tools to, to interact with all of your documents, to do kind of deep analysis, to create documents iteratively, which which are able to pull in information from your from your messages, from your company docs, and so forth. So it's really a way an easy way to bring AI, to your data behind your firewall and then provide all the benefits, productivity benefits, and otherwise to your workforce and to the to your entire company. Jesse, if you don't mind, there is one question that might make that would be capstone to the q and a, and apologies for those questions we didn't get to. But there was a question about what can organizations do to employ cultural changes or other changes to truly embrace command a. I'd actually like to broaden this question to I work with companies every single day. Command a is that unique in the terms of AI adoption. And a lot of what we see is companies preparing for actually embracing generative AI and able to effectively move into production. Often the first decision for companies to make here is how you want employees and teams inside of your company to have access to these different types of models. Having a central function that can help control where employees are being routed to for secure approved Gen AI applications is gonna be crucial to enable your developers and product teams and various organizations to see value from this. In terms of cultural change, it varies by company depending on how far along you've been in embracing new technology. But encouraging employees to leverage this in approved safe manners is gonna be crucial for making sure that adoption is actually carried on going forward. So I'd encourage you to start there, figure out how your company wants to step foot into GenAI, what platforms you want to use to access that, and then slowly roll that out to the company to actually enable product designers and engineers to start building on it. Love it. Thank you so much, David. Thank you so much, Nick, for coming here with myself today to talk. Thank you everyone for joining in. We hugely appreciate it. please do connect with us over on LinkedIn and contact us, on the Cohere website. Hugely appreciate everyone for joining in, and have a great rest of your days..