It seems everyone today is rushing to build yet another infrastructure, yet another Infrastructure as a Service (IaaS) cloud. We all think we are a snow flake more special than all the others but at some point, we need to take a good look around and come to terms with reality. We are not going to be able to build a better, more efficient IaaS infrastructure than AWS. We will not be able to compete in a market space where prices are on a race to the bottom.
We are not going to bend the trend of inevitable consolidation which represents the market clearly signaling that there is no need for many cloud providers. If a company like HP, with its significant financial resources waffles between becoming an IaaS or not, research infrastructures should definitely not employ tax payer money for yet another OpenStack deployment. There is no argument that can stand to justify such course of action. Period.
What makes an IaaS successful? Big upstart pockets, an exquisite implementation of DevOps, an ecosystem of people, organizations and companies developing cool new capabilities and services and yes, some technological innovation that will provide temporary (key word “temporary”) differentiation. If we look at research infrastructures, we hardly have experience in either of these key areas except maybe technological innovation however, even that is limited. The technological innovation needed here comes from experience with operating large infrastructures and not from putting together platforms that support specific scientific projects. This is not to mention that we bet on one cloud automation and then had to change course to where all the industry is going: OpenStack.
Let us step back from the coolness of building our own little cloud and think about our customer: the scientist, the researcher, the student, the innovator. These people and their projects are very similar to startups, what they need is a platform, not another IaaS. In today’s World where you can get modules for virtually every function you need and all you have to do is stitch them up into the service or functionality you need, who wants to deal with yet another IaaS? The research community would be much better served by a platform that leverages the many IaaS options available, that brokers between the researchers and the cloud providers out there who know the infrastructure business very well.
We already have a great asset, the backbone that can deliver access to these various resources. That asset is called GEANT. We already have a unique expertise, the knowhow of managing a federated environment. The European Research Space has the resources and experience to become a Cloud broker for all it constituents. Lets invest smartly in what our customers, the researchers, the academics and students need: Services, Tools, Applications. The alternative is a tax payer money black hole that cannot be justified by any of the fast arguments we are bound to hear: security, performance, customization. All of those are already covered and delivered at much less costs than could be built by the community from the ground up. Moreover, as a broker, as a large community, this platform can drive the IaaS providers and can influence them in the pursuit of specific capabilities and services.
The fastest growing market segments for SaaS adoption are the education and the health sectors. There are plenty of reasons why SaaS is chosen and not IaaS. The facts speak for themselves, let us focus on the more important challenges at higher level of the stack. Let us focus on the users of Scientific infrastructures and their true needs.
Personal views on research eInfrastructures 
I would not compare the merits of IaaS, PaaS and SaaS for their market potential, but rather look at the needs of international research collaborations in Europe and worldwide and at how Open Science can take advantage cloud provisioning.
1. Multi-disciplinary science and international research collaborations generate data and integrate heterogeneous data from multiple existing data sources. A common denominator for all is the problem of getting scalable and efficient access to these data. You can get an idea of how many research communities share this problem from the Open Data Cloud session at the recent EGI Conference (http://go.egi.eu/OpenDataCloud).
The provisioning of a hybrid federated IaaS cloud infrastructure that supports the FAIR principles (Findable, Accessible, Interoperable, Reusable) for European research data is still an open question. This means a federated IaaS environment that ensures sharing and portability of data and data applications in a distributed environment supporting federated ID management, where data can be deposited to provide “Data as a Service”, let’s call this a European data marketplace, to realize the Data Commons. EGI is working to realize this with a number of research infrastructures and other partners.
2. It is not just about improving technology, we have governance aspects to be addressed involving funding agencies, data providers, knowledge organizations, e-Infrastructures, RIs, and most importantly, users.
Open Science is based on various principles, among which are Openness, Participation, Collaboration, Sharing, Re-use to provide greater social value. None of the research initiatives of Europe wants to work in silos, nor the tax payer wants to pay twice. The entire research process needs to be opened, this means shared managed access to scientific instrumentation, computing and storage, data and knowledge. So it is not just about providing PaaS and SaaS for some users, but it is about realizing a European research cloud where these services can be made available as a “European system of services” with EC coordination.
LikeLike
These are all good points and highlight the focus areas I am suggesting as well: Service Catalog, ID management, Data portability, Openness. However, lets move past high-level definitions and goals and step towards implementation with a pragmatic approach and in the spirit of today’s IT World. We often times have the tendency to think our problems are more complex and more important than that of others. In my experience however, you can dismiss these misunderstandings by carrying an informed conversation on concrete terms, around explicit and not generic requirements.
As for innovation, we are in agreement, the biggest opportunity to innovate when it comes to empowering research is not infrastructure, it is governance, enablement of research workflows, it is process innovation.
In my view, many of the components needed to create a pan-European research platform are available today. More collaboration between the key European infrastructure projects would accelerate the integration into a powerful services broker that will address the needs of the target users. Unfortunately, my observation is that we are still building kingdoms around layers of technology, service categories or funding structures. This goes against what the industry has shown to work well for innovation, agility and enablement. For example, it is great EGI wants to become a Cloud Broker but is GEANT at the discussion table? I believe GEANT is running services and functionality that are critical to enabling an agile pan-European cloud infrastructure.
Let me close with another parallel to industry. Everyone agrees that AWS is by far the best cloud provider out there. It all started with Bezos sending out an email to his team that said (loosely quoting here): You will open up APIs for everything and if you don’t, you are fired. Do we have such an unequivocal commitment to interoperability in our European research infrastructure projects? Let us focus on that together with mapping the workflows of our users and I think we will discover that our leggo box is pretty well stocked to do most of the things we want to do.
LikeLike
The point is that the research infrastructure sector collectively needs to engage more strongly with industry and to work with industry to both take advantage of what is on offer and to adapt it, innovate it to meet the needs of research. Taxpayer’s money would be much better spent in this direction than in trying to go it entirely alone, which is what has happened for the past 15 years!
There is a danger is that emerging discipline-specific research infrastructures (i.e., the ESFRIs) will not look to EGI.eu for their e-Infrastructure needs but will go directly to commercial providers. Unless the EC regulates, of course!
My suggestion is that the whole sector should do a deal at continental scale with two commercial cloud providers (thus allowing them to compete with each other) and then focus on building what is needed on top of that to meet domain and discipline specific needs. EGI.eu is reasonably positioned, in terms of understanding the sector to act as broker, governor and systems engineer. But it needs to go onto a commercial footing now. EGI.eu has to become like a SWIFT (banking) and or a SITA (aviation) now.
Project funding mechanisms should more explicitly recognise the need and make provision to pay for infrastructure services. Actually, it’s already an eligible cost AFAIK.
LikeLiked by 1 person
Could not agree more with your comment. Yes, leverage the capabilities, efficiency and lessons learned by the commercial providers to gain access to an efficient infrastructure. Couple with that the purchasing the influence power of consolidation representation. I like the SWIFT analogy as it wraps into the cloud broker concept some more stringency around governance.
You also bring up a very good point: the development and operational funding model for this new IT environment must be worked out. This will be an interesting exercise but ERA has some experience in accommodating a federated structure, nevertheless, it will be another area where we can demonstrate leadership.
Great post. Thanks!
LikeLike