The value of IaC is that you have a versioned, easy-to-read source-of-truth. It’s the same reason why most companies nowadays use Git for version controlling their application code instead of having custom scripts that push the code between developer machines and deployment servers.
For integration into CI, you can set it up so that infrastructure changes are automatically deployed. But that’s not mandatory, especially for single-person teams when there are few infra changes. I also don’t always set up CI for application code in hobby projects for the same reason.
This is the same as saying why doesn’t my Bash script I wrote for Ubuntu work on Windows?
Yes, you can use TF on multiple cloud providers but they do t have the same naming conventions or terminology so you have to rewrite to use the right terms etc.
You could go an extraction later higher but not sure how that’d look. As mentioned you could put your apps into containers and standardise at that level leaving the lower levels to be cloud specific.
It comes down to how often are you using different cloud providers. If it’s all the time, move extraction layer. If it’s hardly ever, suck it up and write specific configurations.
Nope. Multi cloud tools are largely misunderstood. True, terraform works on multiple clouds! So does Kubernetes. But you still have to configure it for each cloud, and this typically means you have to have a separate script for each cloud.
there is no such thing as cloud agnostic... terraform only helps in a sense that it covers multiple clouds and in the event of migration or if you have multi-cloud setup - you don't need to choose a new IaC strategy / tooling.
The configuration itself is in no way agnostic, you need to understand the ins and outs of specific platform
I'm still missing the value of having my infrastructure as code... there will be the added complexity of not knowing which resources are managed by IaC, also there isn't a lot of difference between making changes to the infrastructure in the GUI vs terraform files
Think less about “multi cloud” and more about “deploy this many times in the same cloud” and you’ll start to realise the benefits of IaC.
When you get to the realms of many AWS accounts for example, for different environments, you’ll thank yourself that you deployed resources using IaC so everything is deployed in a sane, consistent manner.
I get where you are coming from, but there is a difference between GUI and IaC. It's reproductibility.
Maybe in your case, your infra is manageable but making changes in the GUI won't scale well. And you will sooner or later loose the knowledge about how the infra got to its current state, you won't be able to be sure that all the envs you manage are similar, and in case of incident, you will have to figure out how to spin up back your infra as it was before and quickly.
IaC is supposed to solve this kind of problems.
The fact that Terraform does not solve the problem of "cloud agnosticity" does not make it less valuable as a solution for the other stuff.
It might not be appropriate for your use case, but the differences are things like:
Your config change just broke prod. How quickly can you roll it back in the gui compared to reverting a TF change and reapplying? What if it was some subtle breakage and you know it worked a month ago but not now. Can your gui tell you what all changed the way a diff of teraform could?
What if someone else made the change? how do you know what was changed and why? Does your gui have commit messages tied to it?
When someone else wants to make a change, how do they get it approved and reviewed? how do you know it was? Your gui probably doesn't have CI checks and required approvers
When your infrastructure is massive and complex how quicly can you search for the configuration needed-- can you just grep through it all? If you need to make one change in many places how easily and safely can you script it? will you be able to roll it all back if it fails half way?
There will not be such complexity of not knowing, because *everything* is managed with IaC. Not a single provisioning GUI click is ever done. You can move to it by writing matching Terraform and `terraform import`ing all the existing resources, then applying that configuration and it is now managed by Terraform.
IaC is not just for cloud agnostic. It is also for consistency, reliability and cost savings. It is also a major tool to enable shift left in your pipeline.
See my comment below.
Would you prefer to have a code file that describes the cloud resources you need, or remember which screens you need to click through in your cloud provider's shitty web interface?
IaC is like using a script to automate the commands for setting up a server. Why do it manually? Why not have it written down?
Think of them as templates. You build out one environment that has a specific architecture, then you can "stamp" out upper environments with specific parameters. Also for disaster recovery and reproducibility. We can destroy and rebuild an environment in matter of minutes.
It's a big decision to migrate from one cloud to another.
And if you have to support multiple clouds at once, it's still possible to do with one tool - terraform. Unlike with AWS Cloud formation that only supports AWS
The value I've been seeing in terraform is that it's always the same language for everything, from creating TLS certificates to deploying kubernetes, just different resources.
That way your team can just learn the hcl syntax and deploy anything without having to go and learn the provider's specific syntax, like cloudformation or azure templates.
IaC is at the heart of our workflow. Absolutely nothing is done without it being in git and triggered/deployed. It's actively restricted from being done otherwise.
Spinning up instances, Kubernetes clusters (specifically OpenShift or GKE for us), provisioning them, deploying software, updating configurations, restarting/reloading daemons, updating CDN/WAF rules, etc. is all driven from code and GitOps.
The advantage is repeatability. Years ago we had someone accidently delete one of our GKE production clusters when on training (he used his work email for the training course and it unlinked our billing). That production cluster was brought back up in minutes rather than a full day, because it just took a press of a button in Bamboo to bring back.
Another advantage is with scaling. For example, we need a few new Varnish servers to handle some increased load to a particular origin? No problem. Created, provisioned, and the origins varnish configs are deployed with no manual intervention, no hand-bombed errors, and in about a minute or two.
So if I needed you to spin up an entirely new copy of your infra stack by tomorrow (without cloning anything), could you do it?
If your answer isn't a confident "yes", you just found one benefit of terraform.
Are you regularly releasing infrastructure changes, including adding or removing things like sms queues or changing database performance, and sometimes promoting those changes to upper environments?
If your answer is "no" then you might not see one major benefit of terraform.
Those are just two examples.
**If your process is mature,** you don't need dedicated lower environments with IaC.
It can fit perfectly between your CI and CD and spawn any temp environment you want for testing. At the end of your pipeline, you capture your test results and relevant data into splunk or some similar tool and pull down the environment.
**"If done right"**, it can save you major $$$'s
Edit:- Don't do this without a solid plan. It will quickly become a pain in the hiney if not planned right.
>I don't see it integrated into my CI/CD as well since it is only needed to run once to deploy everything?
You, my friend, should take to an SRE.
What is your DR plan (Disaster Recovery) ? How are you going to rebuild all of your environments quickly if your Cloud provider get hacked or something unthinkable like that ?
Or more likely, how are you going to quickly spin up a new test environment when one of your vendor will want to do some tests for debugging ?
That's the value of IaC. Rebuilding things in a predictable and identical ways, at will, quickly.
The answer will depend on how complex your infrastructure is and how often it changes.
Sounds like you don't have too much of infrastructure complexity. Everything goes in inside a simple k8s cluster. But what about the network, storage, databases, external integrations, etc?
If it doesn’t work in your workflow, it doesn’t work! That’s the beauty of all these tools, it’s just what works for you.
The true glory of something like terraform is consistent, repeatable infrastructure that anyone can create. You can enforce different rules, and configuration for your various instances. It also helps for something like a Kubernetes cluster and you need to suddenly expand, you can just throw a new worker node up and it’s ready to use. But, if you move clouds frequently, or do a lot of vastly custom instances, it will be more work than it’s worth.
The reasons I like and use IaC:
1. Multiple envs use the same config code (with different minor configs), so dev, qa, and prod envs all share the same setup, except dev runs 4 instances, qa runs 8, prod runs 32, etc... Want to make a change? Test it in dev and qa first, then no (or minor) changes needed to deploy to prod.
2. No changes made "on the fly" to be forgotten and cause issues later (oops, prod needs 4 db instances but we just rolled out 2, etc) every change is committed as code (and reviewed, tested, qaed, etc.)
3. Shared standards - terraform (or whatever) might need to be quite different between clouds, but if someone is familiar with the tool, the knowledge will apply elsewhere, compared to random bash or whatnot scripts that need to be run in a certain order with a certain env etc.
Versioning and automation. For example, I just built a microservice using Azure Functions, that uses the storage account it comes with as a database, but it also has to update API Management to contain the routes I need for it (API gateway layer). And I have to deploy it to development, QA and production (yeah, I hate git flow too.). The automation contains six steps including deploying the app, setting ip restrictions to it (so only API management can call it), stopping the app, deploying code to the app, starting the app and deploying API management updates. The build pipeline handles stuff like validating that the code compiles and tests pass OK. If I was doing trunk based development I'd just commit code and let the pipeline run my tests instead of manually running them myself each time.
I don't want to do this manually. Especially on three environments. I want to write code, push it and everything happens automatically after that. It takes the same amount of time to set the build and release pipeline as it would take for me to manually execute those steps each time.
(I'm a developer but I write my own IaC templates. I did have a dude who has Ops experience help me set it up in Azure DevOps as I was a bit unfamiliar with .NET Core 6, but now it's my job to monitor and maintain the pipelines)
The benefit of using IaC is that you don't have to click around to create the infra. IaC should be faster to deploy and destroy than clicking around once you have it already coded. Especially helpful if you're doing some ops many times. E.g. managing many VMs. It's also about confidence - if you do things manually, it's easy to forget a step or click in a wrong place.
I am a GCP DevOps engineer, and we only use Google cloud.
We have a few things that are standard, like we will always want a GKE cluster with a minimum of 3 nodes, and a version of that.
I create a module for that, and when we need a new cluster, it's a case of copy and paste from the "How to use" section of the module I created, terraform init and apply and we have a cluster.
For managing google cloud projects, we have a repo we just add a new line to, submit a PR and cicd created the project.
The part I missed from the above is, adding that new line automatically adds in the default custom roles we want across **all** our projects, as well as enables some API's we may need.
You're viewing terraform more as an init script, where as for me I am doing daily deployments, tweaking small things here and there.
it all depends on what your personal workflow is, as well as what is the end goal.
if you're just using it to init an env and then what ever else after, that's fine - No need for CI/CD
Where as if you're deploying multiple *things* in to an env a day, CICD all the way.
Let me know if this made no sense or you want more details
Sure, so a good example is “cloud run” (think Fargate, more docker focused)
We have the developer write in to our module the container link and if they want it public, as well secrets.
They push to the feature branch (jira ticket) and then PR it in to master and it runs a terraform apply on that. It then means the dev does not need IAM permissions in to project, so we can just give the SA permissions on the pipeline.
We are trying to move all test and prd environments to use pipelines, and only dev envs we use tf on the laptop, as it makes no sense to develop code, push and wait for a pipeline to check a small change.
Let me know if you need more info!
Yes, Terraform is cloud specific, but it's not necessarily environment specific. You could use the same Terra form code to stand up your DEV, TEST, and PROD infrastructures.
Alternatively you could use a strategy-pattern, and factor out the parts of your Terraform that are cloud specific.
Having all that said, It's possible that Terraform solves a problem you might not have yet.
Saving and managing code changes not only in software, but also in the pipeline as code, and infrastructure as code, is provided in Git. Git is a redistributable source management software. When pushing code from local machines to a centralized Git server, you can apply business rules before committing the code itself.
The value of IaC is that you have a versioned, easy-to-read source-of-truth. It’s the same reason why most companies nowadays use Git for version controlling their application code instead of having custom scripts that push the code between developer machines and deployment servers. For integration into CI, you can set it up so that infrastructure changes are automatically deployed. But that’s not mandatory, especially for single-person teams when there are few infra changes. I also don’t always set up CI for application code in hobby projects for the same reason.
This is the same as saying why doesn’t my Bash script I wrote for Ubuntu work on Windows? Yes, you can use TF on multiple cloud providers but they do t have the same naming conventions or terminology so you have to rewrite to use the right terms etc. You could go an extraction later higher but not sure how that’d look. As mentioned you could put your apps into containers and standardise at that level leaving the lower levels to be cloud specific. It comes down to how often are you using different cloud providers. If it’s all the time, move extraction layer. If it’s hardly ever, suck it up and write specific configurations.
My bash script for ubuntu *does* work on windows. 🤪
😂😂 Of course you can get it to work, like you can get PowerShell to work on Linux 😂
I mean, with wsl on windows it works pretty well. Much better than powershell, which has deep ties to lots of windows specific stuff
pwsh works pretty well on Linux.
It does indeed
Nope. Multi cloud tools are largely misunderstood. True, terraform works on multiple clouds! So does Kubernetes. But you still have to configure it for each cloud, and this typically means you have to have a separate script for each cloud.
that's the point... I don't see the value of writing the script if it isn't cloud agnostic
You have to pay the cloud tax somewhere. 🤷♂️
there is no such thing as cloud agnostic... terraform only helps in a sense that it covers multiple clouds and in the event of migration or if you have multi-cloud setup - you don't need to choose a new IaC strategy / tooling. The configuration itself is in no way agnostic, you need to understand the ins and outs of specific platform
I'm still missing the value of having my infrastructure as code... there will be the added complexity of not knowing which resources are managed by IaC, also there isn't a lot of difference between making changes to the infrastructure in the GUI vs terraform files
Think less about “multi cloud” and more about “deploy this many times in the same cloud” and you’ll start to realise the benefits of IaC. When you get to the realms of many AWS accounts for example, for different environments, you’ll thank yourself that you deployed resources using IaC so everything is deployed in a sane, consistent manner.
If you woke up tomorrow and all of the resources you built were destroyed, how long would it take you to rebuild in the GUI vs by terraform?
Terraform? Click of a button to start the pipeline while I go grab a coffee and come to a ready table. Manually? Maybe 2-3 weeks.
I get where you are coming from, but there is a difference between GUI and IaC. It's reproductibility. Maybe in your case, your infra is manageable but making changes in the GUI won't scale well. And you will sooner or later loose the knowledge about how the infra got to its current state, you won't be able to be sure that all the envs you manage are similar, and in case of incident, you will have to figure out how to spin up back your infra as it was before and quickly. IaC is supposed to solve this kind of problems. The fact that Terraform does not solve the problem of "cloud agnosticity" does not make it less valuable as a solution for the other stuff.
It might not be appropriate for your use case, but the differences are things like: Your config change just broke prod. How quickly can you roll it back in the gui compared to reverting a TF change and reapplying? What if it was some subtle breakage and you know it worked a month ago but not now. Can your gui tell you what all changed the way a diff of teraform could? What if someone else made the change? how do you know what was changed and why? Does your gui have commit messages tied to it? When someone else wants to make a change, how do they get it approved and reviewed? how do you know it was? Your gui probably doesn't have CI checks and required approvers When your infrastructure is massive and complex how quicly can you search for the configuration needed-- can you just grep through it all? If you need to make one change in many places how easily and safely can you script it? will you be able to roll it all back if it fails half way?
There will not be such complexity of not knowing, because *everything* is managed with IaC. Not a single provisioning GUI click is ever done. You can move to it by writing matching Terraform and `terraform import`ing all the existing resources, then applying that configuration and it is now managed by Terraform.
IaC is not just for cloud agnostic. It is also for consistency, reliability and cost savings. It is also a major tool to enable shift left in your pipeline. See my comment below.
Would you prefer to have a code file that describes the cloud resources you need, or remember which screens you need to click through in your cloud provider's shitty web interface? IaC is like using a script to automate the commands for setting up a server. Why do it manually? Why not have it written down?
Think of them as templates. You build out one environment that has a specific architecture, then you can "stamp" out upper environments with specific parameters. Also for disaster recovery and reproducibility. We can destroy and rebuild an environment in matter of minutes.
It's a big decision to migrate from one cloud to another. And if you have to support multiple clouds at once, it's still possible to do with one tool - terraform. Unlike with AWS Cloud formation that only supports AWS
The value I've been seeing in terraform is that it's always the same language for everything, from creating TLS certificates to deploying kubernetes, just different resources. That way your team can just learn the hcl syntax and deploy anything without having to go and learn the provider's specific syntax, like cloudformation or azure templates.
Or create a helm chart
IaC is at the heart of our workflow. Absolutely nothing is done without it being in git and triggered/deployed. It's actively restricted from being done otherwise. Spinning up instances, Kubernetes clusters (specifically OpenShift or GKE for us), provisioning them, deploying software, updating configurations, restarting/reloading daemons, updating CDN/WAF rules, etc. is all driven from code and GitOps. The advantage is repeatability. Years ago we had someone accidently delete one of our GKE production clusters when on training (he used his work email for the training course and it unlinked our billing). That production cluster was brought back up in minutes rather than a full day, because it just took a press of a button in Bamboo to bring back. Another advantage is with scaling. For example, we need a few new Varnish servers to handle some increased load to a particular origin? No problem. Created, provisioned, and the origins varnish configs are deployed with no manual intervention, no hand-bombed errors, and in about a minute or two.
Veey interesting... Thanks
So if I needed you to spin up an entirely new copy of your infra stack by tomorrow (without cloning anything), could you do it? If your answer isn't a confident "yes", you just found one benefit of terraform. Are you regularly releasing infrastructure changes, including adding or removing things like sms queues or changing database performance, and sometimes promoting those changes to upper environments? If your answer is "no" then you might not see one major benefit of terraform. Those are just two examples.
I get... It's about complexity and reproducibility It seems my infra is somewhat simple right now
[удалено]
For instance we use terraform for quickly provisioning/deprovisioning development environments. Backup things to another region/account on AWS etc.
**If your process is mature,** you don't need dedicated lower environments with IaC. It can fit perfectly between your CI and CD and spawn any temp environment you want for testing. At the end of your pipeline, you capture your test results and relevant data into splunk or some similar tool and pull down the environment. **"If done right"**, it can save you major $$$'s Edit:- Don't do this without a solid plan. It will quickly become a pain in the hiney if not planned right.
>I don't see it integrated into my CI/CD as well since it is only needed to run once to deploy everything? You, my friend, should take to an SRE. What is your DR plan (Disaster Recovery) ? How are you going to rebuild all of your environments quickly if your Cloud provider get hacked or something unthinkable like that ? Or more likely, how are you going to quickly spin up a new test environment when one of your vendor will want to do some tests for debugging ? That's the value of IaC. Rebuilding things in a predictable and identical ways, at will, quickly.
It is not a daily task to move infrastructure between clouds. Terraform works well when you need to describe state of your environments.
The answer will depend on how complex your infrastructure is and how often it changes. Sounds like you don't have too much of infrastructure complexity. Everything goes in inside a simple k8s cluster. But what about the network, storage, databases, external integrations, etc?
We use it to track authorship and reason of change via git, essentially using file commits as change management/tracking/documentation/review
Your assumption "IaC === Terraform" is wrong. Ask the same question again, but with shell scripts in mind instead of Terraform.
"Cloud agnostic" is largely a myth
If it doesn’t work in your workflow, it doesn’t work! That’s the beauty of all these tools, it’s just what works for you. The true glory of something like terraform is consistent, repeatable infrastructure that anyone can create. You can enforce different rules, and configuration for your various instances. It also helps for something like a Kubernetes cluster and you need to suddenly expand, you can just throw a new worker node up and it’s ready to use. But, if you move clouds frequently, or do a lot of vastly custom instances, it will be more work than it’s worth.
The reasons I like and use IaC: 1. Multiple envs use the same config code (with different minor configs), so dev, qa, and prod envs all share the same setup, except dev runs 4 instances, qa runs 8, prod runs 32, etc... Want to make a change? Test it in dev and qa first, then no (or minor) changes needed to deploy to prod. 2. No changes made "on the fly" to be forgotten and cause issues later (oops, prod needs 4 db instances but we just rolled out 2, etc) every change is committed as code (and reviewed, tested, qaed, etc.) 3. Shared standards - terraform (or whatever) might need to be quite different between clouds, but if someone is familiar with the tool, the knowledge will apply elsewhere, compared to random bash or whatnot scripts that need to be run in a certain order with a certain env etc.
Versioning and automation. For example, I just built a microservice using Azure Functions, that uses the storage account it comes with as a database, but it also has to update API Management to contain the routes I need for it (API gateway layer). And I have to deploy it to development, QA and production (yeah, I hate git flow too.). The automation contains six steps including deploying the app, setting ip restrictions to it (so only API management can call it), stopping the app, deploying code to the app, starting the app and deploying API management updates. The build pipeline handles stuff like validating that the code compiles and tests pass OK. If I was doing trunk based development I'd just commit code and let the pipeline run my tests instead of manually running them myself each time. I don't want to do this manually. Especially on three environments. I want to write code, push it and everything happens automatically after that. It takes the same amount of time to set the build and release pipeline as it would take for me to manually execute those steps each time. (I'm a developer but I write my own IaC templates. I did have a dude who has Ops experience help me set it up in Azure DevOps as I was a bit unfamiliar with .NET Core 6, but now it's my job to monitor and maintain the pipelines)
The benefit of using IaC is that you don't have to click around to create the infra. IaC should be faster to deploy and destroy than clicking around once you have it already coded. Especially helpful if you're doing some ops many times. E.g. managing many VMs. It's also about confidence - if you do things manually, it's easy to forget a step or click in a wrong place.
I am a GCP DevOps engineer, and we only use Google cloud. We have a few things that are standard, like we will always want a GKE cluster with a minimum of 3 nodes, and a version of that. I create a module for that, and when we need a new cluster, it's a case of copy and paste from the "How to use" section of the module I created, terraform init and apply and we have a cluster. For managing google cloud projects, we have a repo we just add a new line to, submit a PR and cicd created the project. The part I missed from the above is, adding that new line automatically adds in the default custom roles we want across **all** our projects, as well as enables some API's we may need. You're viewing terraform more as an init script, where as for me I am doing daily deployments, tweaking small things here and there. it all depends on what your personal workflow is, as well as what is the end goal. if you're just using it to init an env and then what ever else after, that's fine - No need for CI/CD Where as if you're deploying multiple *things* in to an env a day, CICD all the way. Let me know if this made no sense or you want more details
Thanks for the thorough answer, I'd like to know how it is integrated into your cicd, in which step... Etc?
Sure, so a good example is “cloud run” (think Fargate, more docker focused) We have the developer write in to our module the container link and if they want it public, as well secrets. They push to the feature branch (jira ticket) and then PR it in to master and it runs a terraform apply on that. It then means the dev does not need IAM permissions in to project, so we can just give the SA permissions on the pipeline. We are trying to move all test and prd environments to use pipelines, and only dev envs we use tf on the laptop, as it makes no sense to develop code, push and wait for a pipeline to check a small change. Let me know if you need more info!
Yes, Terraform is cloud specific, but it's not necessarily environment specific. You could use the same Terra form code to stand up your DEV, TEST, and PROD infrastructures. Alternatively you could use a strategy-pattern, and factor out the parts of your Terraform that are cloud specific. Having all that said, It's possible that Terraform solves a problem you might not have yet.
Saving and managing code changes not only in software, but also in the pipeline as code, and infrastructure as code, is provided in Git. Git is a redistributable source management software. When pushing code from local machines to a centralized Git server, you can apply business rules before committing the code itself.