Hello, Spacelift!

spacelift is a specialized, https //www terraform io compatible continuous integration and deployment (ci/cd) platform for infra as code it's designed and implemented by long time devops practitioners based on previous experience with large scale installations dozens of teams, hundreds of engineers and tens of thousands of cloud resources at the same time, spacelift is super easy to get started with you can go from zero to fully managing your cloud resources within less than a minute, with no pre requisites it integrates nicely with the large players in the field notably integrations/source control/github md and integrations/cloud providers/aws md if you're new to spacelift, please spend some time browsing through the articles in the same order as they appear in the menu start with main concepts and follow with integrations if you're more advanced, you can navigate directly to the article you need, or use the search feature to find a specific piece of information if you still have questions, feel free to https //spacelift io/contact do i need another ci/cd for my infrastructure? yes, we believe it's a good idea while in an ideal world one ci system would be enough to cover all use cases, we don't live in an ideal world regular ci tools can get you started easily, but terraform has a rather unusual execution model and a highly stateful nature also mind the massive blast radius when things go wrong we believe spacelift offers a perfect blend of regular ci's versatility and methodological rigour of a specialized, security conscious infrastructure tool enough to give it a shot even if you're currently happy with your infra as code ci/cd setup in the following sections, we'll try to present the main challenges of running terraform in a general purpose ci system, as well as show how spacelift addresses those at the end of the day, it's mostly about two things collaboration and security collaboration wait, aren't cis built for collaboration? yes, assuming stateless tools and processes running stateless builds and tests is what regular cis are exceptionally good at but many of us have noticed that deployments are actually tricker to get right and that's hardly a surprise they're more stateful, they may depend on what's already running terraform and your infrastructure in general is an extreme example of a stateful system it's so stateful that it actually has something called https //www terraform io/docs/state/index html (see what we just did there?) as one of its core concepts cis generally struggle with that they don't really understand the workflows they run, so they can't for example serialize certain types of jobs like terraform apply , which introduces actual changes to your infrastructure as far as your ci system is concerned, running those in parallel is fair game but what it does to terraform is nothing short of a disaster your state is confused and no longer represents any kind of reality untangling this mess can take forever but you can add manual approval steps yes, you can but the whole point of your ci/cd system is to automate your work first of all, becoming human semaphore for a software tool isn't the best use of a highly skilled and motivated professional also, over reliance on humans to oversee software processes will inevitably lead to costly mistakes because us, humans, are infinitely more fallible than well programmed machines it's ultimately much cheaper to use the right tool for the job than turn yourself into a part of a tool but you can do https //www terraform io/docs/state/locking html ! yup, we hear you in theory it's a great feature in practice, it has its limitations first, it's a massive pain when working as a team your ci won't serialise jobs that can write state, and state locking means that all but one of the parallel jobs will simply fail it's a safe default, that's for sure, but not a great developer experience and the more people work on your infrastructure, the more frustrating the process will become and that's just applying changes by default, running terraform plan locks the state, too so you can't really run multiple ci jobs in parallel, even if they're only meant to preview changes, because each of them will attempt to lock the state yes, you can work around this by explicitly not locking state in ci jobs that you know won't make any state changes, but at this point you've already put so much work into creating a pipeline that's fragile at best and requires you to manually synchronize it and we haven't even discussed security yet security terraform is used to manage infrastructure, which normally requires credentials usually, very powerful credentials administrative credentials, sometimes and these can do a lot of damage the thing with cis is that you need to provide those credentials statically, and once you do, there's no way you can control how they're used and that's what makes cis powerful after all, they let you run arbitrary code, normally based on some configuration file that you have checked in with your terraform code so, what's exactly stopping a prankster from adding terraform destroy auto approve as an extra ci step? or printing out those credentials and using them to mine their crypto of choice? there are better ways to get fired you'll say and we hear you those jobs are audited after all no, if we were disgruntled employees we'd never do something as stupid we'd get an ssh session and leak those precious credentials this way since it's unlikely you rotate them every day, we'd take our sweet time before using them for our nefarious purposes which wouldn't be possible with spacelift btw, which generates integrations/cloud providers/aws md for major cloud providers but nobody does that! yes, you don't hear many of those stories most mistakes happen to well meaning people but in the world of infrastructure, even tiniest of mistakes can cause major outages like that typo we once made in our dns config that's why spacelift adds an extra layer of concepts/policy/ that allows you to control separately from your infrastructure project! concepts/policy/run initialization policy md , concepts/policy/terraform plan policy md , concepts/policy/stack access policy md which isn't only useful to protect yourself from the baddies, but allows you to implement an concepts/policy/terraform plan policy md#automated code review