Hello, Jianchao. Welcome here. My name is Marina Iantorno. I am a research analyst at KuppingerCole Analysts, and with me is Jianchao Wang, co-founder and director of AltoSice Cloud Solutions. Thank you for being here. Thank you for joining me.
Thank you. Thanks for inviting me, Marina.
Well, so you know that there are many things going on related to AI, and there are some changes that we expect in the cloud solutions as well. If you had to say, what are the key considerations when you are designing a cloud infrastructure, but specifically for AI applications? What would you say? So what are the considerations?
Yeah, we do see in our customers currently, because everyone is trying to integrate the gen AI application into their existing infrastructure. very few people actually bring a brand new service. So they're trying to bring this gen AI into their existing service, kind of make it efficient, or make it a little bit adjustable, for their use. So what we're seeing in those areas will be, looking into the data pipeline, will be slightly a different data pipeline for gen AI, a data storage per se, and a different set of API to make sure it integrate with current infrastructure and are also efficient enough to adapt it with the new request going through for gen AI, for example, a chatbot and or certain things like that. Yeah.
Well, that's interesting. And as well, I believe that in the coming years we will be talking about something else that we should add and consider. And now that you mentioned some example with your customers, can you share some example of how AI has been successfully integrated into the cloud services in your company or for your clients?
Yeah. In our company right now, we're using something called a rack, and trying to retrieve documents. What we find out, in our daily life is everyone is loved to document, trying to document, which is a good thing. Now everyone documents. We always ask, document what to do. So we did that. The problem is you have so many documentation everywhere. And, there's a different version of it everywhere. So we built a one for ourselves to use basically a rack system to retrieve those documentation, from a vector database. So the first step doesn't involve gen AI at all. So large language model. So you actually, finding the location of the documentation. It helped dramatically already. And then after that you can summarize the meaning of the documentation to give you more kind of a direct answer, which is very successful in our company at the moment. Yeah.
Okay. Okay. Well, thank you for sharing that. Another thing you mentioned that there are, well, some, tendencies, some trends and one that we see is that, the edge computing is on the rise. How do you see the relationship between the edge devices and cloud infrastructure evolving and especially, you know, considering the deployment of AI models?
I think, when the cloud computer comes along, it's very centralized and you don't want to own your, infrastructure. That was the idea. Because it's very easy for a startup to have a pay as you go model to go about business, and when AI comes in now, we do see, I call it like federated learning. So data is the key now. So there are applications, you don't want it to transfer all your data in and out from places, you're talking about another layer of security. So, you can have a federated learning system. Basically you have some data setting in your local environment, let's say a hospital, like in my talk yesterday. So you can have patients’ data sitting in a local hospital and train the model there and pass the model parameters to a local location and then create a global kind of gen AI modeled. So I do see that kind of decentralized model, which where data sits at the edge location where the customer is. So that's where I see it now. Yeah.
And, looking towards the future as well, you know, people always want to know about the trends, what will happen, what to expect. And, well, AI and the generative AI models are creating a revolution, I would say nowadays in almost all the spheres. And I believe that in cloud solutions as well, as you say, so then this is something that is impacting. What are the emerging trends in AI and cloud computing that do you believe would be the most, relevant or impactful in the coming, let's say two years?
So when we look at the big company in racing, who has a better gen AI when the models getting bigger and bigger and bigger. I do see another trend for everyday use for people, [...] small and medium company. That's our company facing, ideal customer needs. So what we're seeing is we're seeing those small models will get more and more efficient. And in layman's words, they get smarter, for sure. And you don't need a huge, large model for something we do, like we do the rack, I was just mentioned. That was running a 7 billion parameter model. So you can basically run a mac M1 with 16 gig. So we keep it completely local in our environment. So I do see the smaller parameter model will get popular, along along the normal use, everyday use, as well as the model will get bigger and bigger, as well getting smarter. We'll see some amazing things in the future.
Well, I have a last question for you. And you just mentioned that now you can deploy models locally. Do you think that people will actually stop shifting towards cloud or hybrid methods? Because I can see that in the last years, many companies just shift the work out. Do you think that it will change?
I was, I would think, and that probably won't change. And we're talking about internal stuff probably you can use, let's say set up a little server because customers are really concerned about their business data. That was the whole reason we're building a smart rack within our company. So you don't need it to take care of that level of security. When you transfer data or talking those data or move those data to gen AI, come back all that. But if you're serving customer, you still you need that large scale of infrastructure to, if you're e-commerce or something, you still need that large infrastructure. I don't think you can a hold, it will be very expensive for the initial investment to have that on your local location, so.
And that makes sense. Jianchao, thank you so much for being here today. Thanks for your insights. And well, I'm looking forward to meeting you in the next EIC.
Yeah. Thank you.
Thank you so much.
