Name: Re-shaping QA and DevOps with AI: Insights from Industry Leaders
Uploaded: 2026-01-28T17:16:21.194Z
Duration: 49 min 37 s
Description: Re-shaping QA and DevOps with AI: Insights from Industry Leaders

Transcript for "Re-shaping QA and DevOps with AI: Insights from Industry Leaders": Hello, everyone, and welcome to today's webinar delivered in partnership with OpenText and Cognizant. My name is Tom Chapman, and I am the senior editor of AI Magazine, Technology Magazine, and Cyber Magazine. Today's webinar is titled reshaping QA and DevOps with AI, insights from industry leaders. So what exactly are we going to be discussing today? Well, in an environment where digital expectations continue to rise, organizations are under intense pressure to deliver software that is not only faster, but more resilient, secure, and scalable. Traditional approaches to quality assurance and DevOps are no longer enough to keep pace with demand. AI is now playing a defining role in how enterprises design, test, and deliver software. From intelligent test automation to predictive analytics and continuous optimization, AI is quite simply reshaping the entire software delivery cycle. Over the next forty minutes or so, we'll explore how AI is being applied in real world QA and DevOps environments, the tangible value it's delivering, and what leaders should be thinking about as these capabilities continue to mature. Before we get started, I'd just like to briefly mention AI Live, the London Summit, taking place later this year in October, where there'll be plenty more fascinating conversations very similar to the one we're going to be having today. So lots to talk about and not too much time to do it in. I'm delighted to be joined today by experts from OpenText and Cognizant who will share practical insights, lessons learned, and their perspectives on what the future holds. First off, I'd like to welcome Yaniv Sayers, fellow and ADM chief architect at OpenText. And we also have Shantanu Sengupta, q e and a technology, COE, and head at Cognizant. Yaniv, Shantanu, I hope I didn't butcher your titles too much. Some long ones for me to to wrap my tongue around there. Thank you so much for for being here today with us. Before we jump into the discussion, perhaps you could start by giving a short introduction to yourselves. Tell us a bit about your your careers and, and your current roles. Yaniv, let's start with you. Yeah. Thanks, and and happy to be here. My name is Yaniv Sayers. I'm a fellow and and chief architect for the application delivery management unit in in OpenText. In this capacity, I'm responsible for technology and innovation strategy with clearly a lot of emphasize in in AI, in general, that AI specifically. And, Shantanu, great to have you with us here today. Thank you, Tom, and thank you for having me here. It's a pleasure. And hello, everyone. I am Shantanu. I have been in the industry for more than twenty five years. I started my career as a developer and then transitioned into a full time quality engineering professional. In my current role, as part of the technology office of quality engineering and assurance practice, I lead the r and d around AI and its influence in on on quality engineering. We we develop solutions and work closely with our delivery teams to ensure that they embrace all the best practices and I and IPs to bring all the best for our clients. Fantastic. Well, as I say, thanks so much for for sparing the time to to join us today. Well, let's let's plunge right into our discussion. Shantanu, I'll start with you if that's okay. I just wanna kinda set the scene a little bit for discussion. How are you currently seeing AI fundamentally change the role of QA and DevOps within large enterprise today compared with, let's say, two or even three years ago? Great. Yeah. Absolutely, Tom. So so far, all along, right, the the QE narrative was all about automation. Right? So we we talked about stuff like an automation first approach and so on and so forth. And in that context, we talked about web automation, API, database, in sprint automation, and so on and so forth. Right? So, traditionally, if you look at speed has always come at the cost of quality. Right? So what I what I mean by that is if you try to go faster into production, things things tend to break. Right? But if you try and do thorough testing, you you tend to slow down the SDLC. Right? But as we all know, today, speed is of obvious sense. It is the currency. Speed is the currency. Right? And hence, the traditional automation led QV approach isn't adequate any longer. Right? You you can't be coping up with the the desired release velocity, what your business wants you to deliver. Right? And also, you end up having a large number of test cases, which leads to a maintenance headache, and it's it's an essentially a nightmare to maintain a huge set of test cases. Right? And what what you end up doing is you uncover defects pretty late in the cycle, which isn't desirable. Right? But now with the the power of generative AI and agent API up our sleeves, Qiwi can really transform from being gatekeepers to being quality guardians. Right? And and then, you know, the reimagined AI augmented quality approach allows you to meaningfully shift left. Right? So, essentially, you can start generating your test cases automatically as soon as the requirements of the user stories are are defined or they are checked into the source of truth, maybe Jira. Right? It also enables you to also meaningfully ship. Right? Right? So what I mean by that, it it is it it gives you the ability to analyze production user and data patterns that we test what is reality. Right? Scenarios which actually real users are using in in in production as opposed to testing almost everything under the sun. Right? And what we also see, Tom, today is with AI is is not only a productivity lever. No. It it it doesn't only give you efficiency benefit. Right? That is enables you to do more with less. That is that is not the end of the story. It is also an important quality enabler. Right? We have seen that with AI, you can generate better coverage. You can write better edge cases. You can write more number of negative scenarios and so on. Right? So but in all of this, you will obviously have to ground your AI or your agents to the right level of context, which is which is extremely important. Right? Your your enterprise tribal knowledge is extremely important to be attached to to the AI for you to kind of adopt AI in a in a in a in a more holistic way to get the real benefits out of it. Now I talked a lot about some of the few use cases on the DevOps side. If you see with AI ops, you you can have AI ops to kind of have fewer deployment failures. You can have things like automated anomaly detection and faster remediation. Right? So if if you ask me to summarize, right, so enterprises, previously, they faced scalability hurdles with their QE and DevOps functions handling lot of these repetitive tasks manually. Right? But now with AI, you are able to find bugs faster. You are able to predict where defects will happen even before you start testing. Right? Which Which which enables you to go to production faster, reduce defects, and hence, lead to superior quality and customer experience. Fantastic. Shantanu, thank you for that. I think I think that is the scene most certainly set. Yaniv, what about your thoughts on that? Are you seeing something something similar on the ground? Yeah. I would actually to emphasize what Shantanu just said, I believe that, what we see now with AI is actually disrupting the entire SDLC. And, as, you know, the initial adoption of AI in development actually means that there is an influx of of software changes, of AI generated code. And for QE to to keep up, to avoid having a bottleneck, in the rest of the, let's say, software delivery, pipeline there, that that's in in in QA, in DevOps, in the deployment, you have to use AI. You have to use AI to generate the test, to automate the the execution, the deployment, etcetera. So it actually becomes imperative for QE, for DevOps engineer to leverage AI. And that, in a sense, will redefine the role of QE, of DevOps engineer. They would become the context provider for the agents. They would become the ones that needs to supervise the agents to be to train the agents and and and provide them the feedback. So that is imperative. That is imperative. Otherwise, the rest of the pipeline would become a bottleneck. Risk will risk will evolve. So that is that is a key thing. AI becomes imperative to be used across the life cycle, not just by developers. Great. No. It's a fantastic point, Yaniv Sayers. Thank you. I'll stay with you, Yaniv Sayers, if that's okay. We, of course, see many organizations, countless, in fact, experimenting with AI, but perhaps struggling to move beyond those those pilot projects. So to you, what does successful scalable adoption of AI in QA and DevOps actually look like in practice? Yeah. I think what we see now is a lot of organization, as you mentioned, incubating, experimenting, and somewhat struggling both to scale and realize the ROI of AI. And some of them struggle with a variety of reasons. One is that, you know, not always the generic large language model are sufficient. In many cases, you have your domain specific knowledge, your domain specific know how, your specific application a specialized way of working. So you have to, I would say, leverage that specialized knowledge, your organizational knowledge, industry specific knowledge, application specific knowledge, and integrate that with the generic LLM. And there are different techniques for achieving that, whether that is through retrieval augmented generation and so on. But the key thing here is that you have to connect the dots of the knowledge and information across your SDLC from the source code management to your planning to your issue tracking system, quality management system, etcetera, because this is where the, you know, knowledge is the bloodstream for AI. You have to connect these dots into a knowledge hub that will eventually be able to leverage to provide and generate that optimized context for humans and AI. So this is a this is a basic thing that I see organizations struggle with. How do you get that key information together so it could be harnessed and and and be the source for AI to get that, I'd say, responses and optimized experience. The second thing is that in many cases, AI is being used initially by developers. And as I mentioned earlier, if you don't integrate and leverage AI across the SDLC, across the life cycle, then you generate bottlenecks. So your QE becomes a bottleneck. Your deployments becomes a bottlenecks. So you have to look holistically across the software delivery life cycle, how do you inject and leverage AI in each and every step, whether that's AI powered planning, AI powered development, AI powered testing, AI powered delivery. It cannot just be around development and and and code reviews. It has to be streamed. It has to be funneled across the value stream, across the life cycle. And I would say another aspect or I would say a challenge is that we see as organization start to implement AI, and they put in place MCP, an agent to an AI to AI integration. Suddenly, the the volumes increase. There is suddenly an an an an uptick of volumes, of load, of workload that is generated by AI, and not always they have their platform engineering tooling ready for that spike, for that additional load. So I would say to be AI ready, you first need to validate that your infrastructure, that your platform engineering tools and systems are ready. And lastly, the the human factor. So the human factor, it's not just the human in the loop approach. The human factor, the engineers, needs to be ready to leverage AI. So initially augment the day to day task, but eventually realize and learn how to make the most out of it, how to, as I mentioned earlier, become kind of a manager of your agents to provide them the feedback, to provide the context, and by that, continuously improve. Some fantastic advice there, Yaniv, I've I've gotta say. Shantanu, anything you'd you'd add at all there when it comes to successful, scalable adoption? Yeah. I think Yaniv Yaniv touched on all the important points. Right? So and then the AI adoption is no more optional. Right? It is it is mandatory to be successful to stay ahead of the curve. Right? So three or four things from our perspective that we see with our clients of with respect to impediments, with respect to scaling and industrialization. One is, obviously, the skill transformation that is needed across your enterprise. Right? So that is extremely important. Second is AI cannot do magic on its own unless you feed the right context. Right? So a lot of lot of organizations, they don't provide the right context of data to the AI engines. Right? And that is where adoption becomes a challenge. The benefit becomes becomes a challenge and and and so on. Right? And, also, you've got to kind of in a in a in a human and AI world where they will coexist, the the role of the quality engineer or for that matter, the DevOps engineer will will change. It will less it will be less of a doer and more of a validator. Right? So that is that is the kind of a mind shift mind mindset change that needs to happen, essentially. Right? And there has to be a a very, very solid governance layer as to with with with proper controls, guardrails, and and then so on and so forth. Great stuff. Yeah, Yaniv, I'll come back to you with with this one. How would you say AI is improving software quality and reliability, particularly in those complex hybrid or cloud native environments? Mhmm. Yeah. So so this kind of environment, the complex environment with high scale, with, I'd say, a significant interdependencies and components tend to be overwhelming for for any human. And with AI, there are several aspects that can be leveraged and improved to improve the quality and reliability. I would say it from the get go is the ability to grasp and focus on the main risks areas and and predict where are the, I'd say, the hotspots. Where are the areas such as a single point of failures? Where are the areas that are less obvious for the human to identify as, as a risk? So you can focus on that. So that is the first thing here, identifying and focusing on the on the key key risk areas, especially in these pretty complex environment and and architectures. The second aspect is leveraging AI for improving coverage. So with AI, being able to generate the test, generate the scenarios, automate the execution so you can actually validate that you that you cover your application, that you cover these complex scenarios. You can scale that. That is a key that is a key aspect that AI can be leveraged. And in addition, I would add that these systems tend to have huge volumes of data, of telemetry, of information. And and here, again, AI can be leveraged to identify issues, be proactive, and in some cases, predictive, realize where there could be potential failures, where there could be potential bottlenecks and issues before they impact productions or end user so you can be proactive and resolve that upfront. Great stuff. Yeah. I think that proactive piece is, is hugely relevant and hugely important to raise. Shantanu, any any thoughts you'd, like to add on that one? Yeah. I just want to add maybe a lot of our clients. Right? They are they are going through massive legacy transformation projects. Right? I mean, they are they are kind of modernizing not only the front end, but the back end legacy systems as well. Right? And you were talking about systems which are, like, twenty years, thirty years old, very less documentation available. Right? So you would struggle in in doing the reverse engineering. Right? So AI AI comes to the rescue. Right? You have the ability to kind of scan through the legacy code and make sense out of what the application is actually doing, generate documentation. Right? And then use that to forward engineer your modernized application. Right? And maybe generate test cases and and all of that. Right? So we we see a lot of that space getting simplified. The the legacy modernization space, it's getting simplified. We believe it is okay. Right? So we just wanted to add that point. Absolutely. Yep. Thank you so much for that. Shantanu, I'll I'll stay with you. From your experience, which areas of of the QA and and DevOps life cycle are really seeing the fastest returns from AI? And, and, therefore, maybe where should organizations be prioritizing their investment? Sure. Sure. Excuse me. So, Tom, really, with the work that we do for our clients, we we see three types of use cases which has an influence to kind of eliminate waste or enable hyper productivity across the SDLC. Right? First, what we call the work companions. Right? So these are essentially your your task performance. Right? So they can retrieve or reference contextual data, and then they can generate test plans, test cases, test scripts, help in doing impact analysis, help in doing defect triaging, and and so on and so forth. Right? So that is they are they are able to do most of the heavy lifting in in in the life cycle with the with lot of precision and consistency. Right? So that's the first category. The second category is what we call the knowledge companions. Right? So, essentially, they provide insights. They can refer to your enterprise tribal knowledge. It it can reference they can reference the industry benchmarks. It can reference your defect patterns and help in decision making. Right? So, essentially, they're they're the backbone of the context engineering process, and they ensure that your your work companions that I talked about, they're grounded to the right data and and and and enable them to provide meaningful output to you. Right? The third piece is is the quality guardians. Right? So, essentially, they are the auditors. Right? So they monitor your environment. They flag anomalies. Right? And ensure compliance, essentially, which which kind of safeguards quality. Now to give you an example of a quality guardian, right, I can talk about the user story value. Right? So what what what can happen is as soon as your requirement or user story gets checked into, say, a source of truth like Jira, you you have the ability to analyze the user story for completeness, conciseness, and so on and so forth. Right? And what what do references you reference your enterprise knowledge in in the form of BRDs or functional specs or web themes, and you you use all of this knowledge to enhance or augment the user story. You add more details. You add additional acceptance criteria and then so on. And since the user story is the entry point for the QE, right, an early intervention, it it ensures better coverage, effective generation of negative scenarios, and so on and so on and so forth. Right? So, essentially, what you are doing is you're driving quality upstream, and you are also having an influence on how code is written. Right? Because you are you are augmenting the user story so the developers will be able to write better code. Right? So so that's it. That's about it. Now let me give you a couple of real world examples of the kind of work we are doing with our clients in this space, Tom. Right? So for example, a leading satellite television provider, they they embarked on a scaled AI implementation across various agile parts. Right? And and we had used cases like test case generation, test script generation, and it led to a productivity improvement of about 40 to 50%. Right? So it it ensured that the TV velocity increased, right, what what the business wanted to look at. A story from an insurer, it they had a completely different problem. Right? So they were migrating their automation scripts to a different framework, and there was a lot of technical debt. In the sense, they had more than 15,000 automation scripts in the in the old framework. Right? And it was a humongous effort which was was estimated to do this migration. And and then AI came to the rescue, and it cut down the migration effort by more than 50%. Right? And if you look at the DevOps side as well, I mean, there are there are use cases like you can analyze logs across the various application peers. You can correlate events, and you can pinpoint root causes for incidents, thereby you can reduce your mean time to resolution. Right? And if you do all of this effectively, your your partner ecosystem, I mean, comes into play. It it it plays a very, very important role. Right? Because it it is a very steep esco when it comes to AI. Right? We we have newer models, newer tools almost every day. Right? So a strong partnership strategy, it gives you the much needed competitive advantage to move forward from. Fantastic. Shantanu, thanks so much, of course, for for those examples. Yaniv, coming back to you, anything to add there? Yeah. Maybe maybe to generalize a bit, when I'm seeing on you know, where where do you have most payback from from AI is in areas where you have, like, huge volumes of data or cases where you need quick decision making, where that could be somewhat overwhelming for humans, and this is where AI can complement or or address. So following on the example from Shantanu, clearly, you have now, like, influx of software changes that you need now to test and validate, then, yeah, automating that test generation by AI is is is definitely paying back. If now we have a change that breaks your test, so having self maintained tests by AI definitely pays back. If now you need to execute the test cycle, being able to run, like, risk based testing and smarter test selection is is paying back. When you have, like, a a production issue and you have enough tons of telemetry data, using AI to finding that needle in a haystack, that is a great payback. So areas where we have high volume of data or quick decision making or, let's say, a a low latency, these are areas where we see very very quick payback from from AI. And and, Yaniv, I I think we're talking more and more these days, aren't we, or so it seems, about the cultural or organizational barriers, that are maybe associated or have to be overcome, when, you know, in AI adoption. Which of those barriers do you really feel that organizations need to to overcome to fully realize the value of of AI driven QA and and DevOps? That that's a that's a that's a big topic. And and I think the the main thing is trust. Whenever we engage with, you know, with our customers and supporting them in their voyage in leveraging AI, in faster software delivery, improving quality, etcetera, It's always the the key thing here, the barrier is is the trust. Eventually, AI and it is. It's it's it's a black box, eventually. It is a black box and in generative AI specifically, from the get go, by design, it is nondeterministic. It has a nondeterministic behavior. It can hallucinate. There are, I'd say, new attack vectors and risks area around privacy, around biasing, toxicity, etcetera, there is higher risk from the get go. And this is a barrier, but there are ways to treat that. And there are ways to manage the risk. There are ways to build trustful trustful AI. And that is achieved both by implementing specific, I would say, methodologies and the ways of working and governance and control of AI. So, for example, explainable AI, that means every interaction with AI has to be explainable. Their decision making, the decision tracing has to be audited. They have initially, they have to be approved and validated by a human. That could be human in the loop. So this is how you build trust. So you start small. You implement it for a few use cases. You learn. You provide the feedback. You add the controls, whether that's a better auditing, testing, validation, etcetera, before you scale that. So that is is key. And there are new methods and and and ways on how to validate and how to test and evaluate the nondeterministic behavior of AI, and that requires new statistical methods to define a baseline for your golden dataset, compare that over time so you can identify what's called kind of a model drift, etcetera. There are ways to mitigate that and build that trust. I would say that is a key key thing. The second the second, I would say, barrier is the the human skill. And as I mentioned, I think we mentioned earlier, the role of an engineer is is changing. You, in a manner, become a manager. You have at your disposal an army of agents. That could be a digital core developer. That could be a testing agent, a performance engineering agent, a security agent. You need to realize how you manage them, how you provide them the goal, how you provide them the context, how you validate the outcomes that they provide and provide them the feedback. So that is a new skill that engineers, managers would need to be equipped with so they would be AI ready. Fantastic. Incredible, really, you know, that concept of, you know, if we went back a couple of decade decades or so, that concept of managing a team of of AI agents, you know, how how times have changed. Shantanu, trust, human skills, anything further to add on on those barriers that that we need to overcome in this space? No. I think a lot of clients we have seen with the initial roadblock or or or or the it roadblock is essentially with respect to security. Right? And this this concept about data sovereignty, right, is is is is is my data leaving leaving my estate, right, or even going out of country, right, with with whatever models you are bringing in, whatever architecture, all these lag models and agent API. Right? Am I am I kind of leaking any PII data, or is my data leaving my my perimeter, right, or or out of going out of country? So those are very valid concerns. Right? And then we we normally get into discussions with the AI board to kind of explain them that how it will work out and then try and kind of calm them down that I mean, the with the controls and the guard rails that we put together. Right? Some of those challenges will be addressed. Right? But pretty much everything Lenny covered. Great stuff. We've already we've already mentioned that word trust, of course. But, Shantanu, as as AI takes on more responsibility in those testing and and release decisions, how do you feel leaders should really think about governance, trust again, and and risk management? Yeah. Yeah. That's a very, very important aspect, Tom. So governance, trust, and risk management, they're they're very important, yeah, imperatives, and they're fundamental building blocks for scaling your AI initiatives. Right? No no doubt about it. Right? So so one one thing we all have to understand that if you are leveraging AI, right, you cannot be delegating the accountability saying, hey. AI recommended, so I'm not responsible. Right? The legal liability cannot be assigned to an algorithm or a machine. It has to be human. Right? So that that's the fundamental thing. So, essentially, while AI is providing you the prediction or the evidence, the humans will have to provide the judgment and and the final sign off. Right? And we have to put together a very, very robust governance framework. Right? And you should you should look at some of the following aspects, starting with the data governance. Right? I I talked about the PII data, leaking of PII data. Right? So you cannot be putting PII data into the models or into the prompts. Right? So input data security is is is extremely important. The second one is Yaniv Yaniv briefly touched upon it is with respect to governance of the AI models. Right? So aspects like drifting of the model, hallucinations, they'll have to be looked at. And and very important is the explainability, right, of the AI. The AI should offer a chain of thought reasoning. Right? For example, if it is a classification algorithm, it it shouldn't just say yes or no. Right? It should also mention the conditions against which it it evaluated and passed on a result of a yes or no. Right? So that's important. The third piece is with respect to release governance. Right? So you will have to verify the output for a period of time with the human in the loop before you consider giving the agent more autonomy. Right? And once you kind of put your agent in an auto mode, you also have to have something like a kill switch, right, which is your callback option in case your agents kind of deviate. Right? You have to kind of have a callback mechanism to kind of change the workflow back to a non AI mode. Right? So all of these are very important in each of those. Right? So, finally, as as organizations, they they are shifting their focus of just leveraging AI for as a productivity LPLC productivity lever, right, they're also trying to kind of reimagine their business processes by infusing AI into their application. Right? And hence, assurance of AI becomes very important in in the agent development life cycle world. Right? And then as as you know, it's it's it's a moving target. It's it's like security thread vector. Right? Once you validate something, your application from a security standpoint, right, since the security thread vectors evolve, right, the security validation has to be a continuous process. And same applies for AI because of the changing landscape of regulations with the various data conditions against which your AI is exercised. Right? So and then how do we do it? Right? We we do it with by leveraging LLM as a judge, or you can also use SLMs, small language models to do all of this validates. Right? So these are some very important aspects that are organized with me to look at. Fantastic. Thanks, Shantanu. And Yaniv, your thoughts on that approach to to governance, to. trust, and risk management? I I think in one hand, there is always, as as we mentioned, that challenge in in trusting AI, but on the other hand, you have to. Eventually, you again, you become a bottleneck. So in a way, you have to use AI beyond automation. It is part of the decision making. And now it's more how do you how do you kind of translate your way of decision making into policies, into examples so AI can learn from and execute upon? So for example, if following on what Shantanu mentioned, when you're looking on risk, so what do you define what do you consider, like, as low, medium, high risk of changes? You could say, yeah. Yeah. Look into the test coverage. Look into whether this is a mission critical system or not, if it's inbound or outbound, if it's, you know, testing or staging, etcetera. So try to mimic that into a policy, into the knowledge that you that you feed AI, And and that that is imperative. So in no matter you teach your AI assistant, the decision making, how to make or qualify a risk, how to make these these decisions. And as Shantanu mentioned, eventually, the human is accountable. You are accountable. You use AI, but you are accountable. So if it fails, you have to review the failure and update the policy. If it's successful, provide that feedback. And over time, it will improve. This is an continuous learning mechanism. Great. Absolutely. Yeah. No. Some sound advice there, Yaniv Sayers. It's difficult to have this conversation without really kinda looking ahead as to what's what's coming next. Yaniv Sayers, what looking ahead, what emerging AI trends or or capabilities, you know, if we haven't mentioned them already, do you see as having the biggest impact on on software delivery over the next, Yep. again, let's say, two or three years? Yep. So I I think we we started with that. I mean, today, most of organization and most of the usage of AI is augmenting the human and and and focusing on on automation. And that means I'm a developer. I will now use my coding assistant, not just for code completion, for code generation, but it is you know, I manage it. Yeah. I I I manage it. The human is the driver. I'm a quality engineer. I will use an agent to generate the test, to review the test, to execute the test, etcetera. But the human is the driver. What we'll see more and more is having the AI as a driver, meaning the human, the shift to autonomous testing, autonomous delivery. The human will define the goal that could be, hey. Here is a new feature we would like to be implemented. That is the goal. You provide the goal. You provide the perimeter. You provide the constraints. You provide the policies. It has to meet that security bar, quality bar, etcetera. But you define the goal, and eventually, the agent, the AI, would go and implement and iterate that with your feedback up to achieving that goal. So that is a that is a major shift, having the human more as the supervisor, as the context provider, as the one defining the goal, and eventually the AI as the executor. Second is the collaborative team of agents. So it's not a single agent. Eventually, there are different specialization required. It's a different specialization for the quality engineering agent, for security agent, for performance engineering agents, etcetera, to the developer agent. So they will need to collaborate together. And they will need to be powered by your organization's specialized organizational knowledge. So just the generic vanilla LLM are good to a certain level. But if you are to avoid hallucinations, if you want a fit for purpose solution, you will have to implement your knowledge hub. You will have to have your knowledge graph to connect the dots so that could be the source the source of knowledge for your agent and and humans. And I would say, lastly, we see the shift not just to AI generated code, but eventually to AI powered applications. So every organization now is providing to their customers, to their end user, not just a natural language experience, but agents. Agent that they could use, whether that's in, you know, online banking and loans and financials and and health care, etcetera. And these agents are the new applications with new experiences, and they will need to be evaluated, tested, and quality assured. And that requires new methodologies, new practices to validate AI powered applications. As mentioned, they are nondeterministic. They are not necessarily consistent. There is a new attack vectors that you will need to assure that requires dedicated methodologies and dedicated implementation practices such defining golden datasets, practices such as defining your baselines and continuously validate regressions upon that baseline and keep managing that datasets. Using LLM as a judge, new practices, new approaches that will have to be implemented so you cannot just validate and assure AI generated code, but also AI powered applications. Fantastic. Shantanu, I can see you nodding along to a lot of what Yaniv had to say there. Is there anything you want to add as we look ahead to the next couple of years? it. I don't believe you for a second. Shantanu, your thoughts. Yeah. No. No. I think Yaniv said all the right things. Right? So it's I mean, this this this models and and these tools, I mean, are getting are getting more and more powerful every single day. Right? So what was unimaginable yesterday, right, is becoming a reality. Right? So, essentially, you you you move to a to a a world of more autonomy from automation. Right? And then you you just have to learn working in a in a in a hybrid ecosystem with the humans with with the digital workforce, right, to kind of do wonders in in SDLC, which was unheard of or uncomfortable. Great stuff. Now, guys, we don't we don't have a huge amount of time left. I just wanna go to one last question, and then we're gonna, if we have time, move on to to some audience questions. Possibly only time for for one in the end. But the final question for me, really, is just finally, what practical advice would you give to tech leaders who are just beginning their journey towards AI enabled QA and DevOps? Shantanu, let's let's start with you. Yeah. Sure. I'll I'll I'll briefly talk about this, Tom. Right? So first and foremost, I think a lot of times we we spoke about context. Right? It is nonnegotiable. Right? You have to have your context. Right? Otherwise, you you can't get AI you can't get the benefit out of AI. Right? So that's one. Second is the skill transformation. That is absolutely essential. Right? You your workforce has to have the foundation and knowledge of AI. Right? And I'll give you an example. In in 2025, Cognizant at an enterprise level, right, we we had a high coding event. Right? And we had more than 200,000 associates kind of trying their hands on different code assistance and AI tools. Right? And we happen to kind of do a Guinness World Record for the largest Genii online icon. Right? So it was a very, very good event. I mean, it helped him the skin transformation process to to a great extent. And since this is a space of very increased flux, right, and there is a steep s curve with respect to evolution of AI, right, there has to be very, very effective leverage of your partnerships. That's that's extremely important. Right? And then you don't have to boil the ocean. You can even start small. Right? And you can start leveraging the code assistance to begin with to get some basic productivity and then kind of go to a platform laid approach to bring in greater consistency. You can you can kind of think about prompt libraries as well to get some degree of consistency. And last but not the least, we spoke about a plan, governance, and observability. Right? We will have to kind of put a have a close eye on what AI is doing in production, right, and take the course corrections as as you as you move on. Right? So if you do all of these things, right, you should be able to kind of achieve your broader goals such as reducing your regression cycles, improving release velocity, reducing defect leakage, and creating an experience for your clients, which is memorable and superior. Fantastic. And Yaniv Sayers, just coming back to you as we any parting advice you'd give to tech leaders? Yeah. Just to follow on on Shantanu, eventually, knowledge is the bloodstream for AI. So the whatever LLM you use, the AI is good as the the context you provided. So build your knowledge hub. I mean, connect the dots, connect the information so it can become available for for your AI. And and and second, take it agile. Like, with any change, with any disruption, with any new technology, start small, incubate, learn, realize the risk, manage it before you scale before you scale across the organization. Fantastic. Awesome, guys. Thank you so much. A fascinating conversation, and we do just about have have time. I'm just gonna pick out one of the one of the several audience questions that we've we've come in as I just gone over to those now. So one that we've had, and I hope I'm pronouncing this correctly, from Xiao Zhang. Shantanu, I'll come to you with this one. When we talk about a how I presume this should say how AI is leveraged to create test cases, predict where the defects most likely occur or where the bottleneck is, does this AI have to have domain specific knowledge? Are you able to tackle that one for us? Yeah. Yeah. Yeah. Absolutely. I mean, you should you should give, like, the I think we we stressed upon the importance of having the right context for your AI to kind of generate something meaningful. Right? So if you if you if you want to generate test cases, right, you you should have so the entry point for a test case in the user story are a requirement. Right? But that's not enough because the user story will have very little information more often than not. Right? So you have to have additional context paid into the AI. Right? If your historical user stories, test cases, and any associated documentation, for example, your business requirements, dysfunctional states, or even even wear frames. Right? So this will be appropriate context. When it comes to defect reaging, right, or helping to fix defects, that is where quality engineers can add value and provide adequate information to the developers. Right? That is where also you need historical defects along with the the the root cause of the historical defects, right, so that you can you can kind of predict the root cause and provide some additional information. You you you can also kind of hook onto your observability tools and get relevant log excerpts and then and then attach it to the to the defect, right, so that the developer has the detailed information to kind of fix the defect in an accelerated manner. Great stuff. And Yaniv, if it's okay, I'll just come to you with one more question from Yatesh Kateria. With this shift to AI powered applications that you mentioned earlier on, do you see the boundaries between software and services diminishing? I think that I'm not sure if diminishing, but I think they will need to collaborate even further. Because eventually, as mentioned, data is the bloodstream. You will need to get that production data so you can then build on top, learn from it, test and evaluate, and so forth. So the collaboration between the the different functions will be even more important as AI is, I would say, data driven. Data driven mean that the data spans the life cycle. It's from the business planning information, from the quality management, from the operations team, etcetera. You have to connect the dots. You have to share that information so you can get superior results. Awesome. Great. Yaniv, that's a a fantastic way to end, I'd say. Yaniv, Shantanu, I'm afraid that is all we've got time for for today. My thanks to all of you for watching, and Yaniv, Shantanu once again. Thank you so much for joining us today. We'll we'll speak again soon. No doubt. It's been a it's been a great conversation. Great. Thank you. Thanks. Thank you all. Thanks so much, Thank. you all. Now