You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/blog/2022-12-01-multicluster-devsecops.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -74,7 +74,7 @@ While these pipelines are included in the pattern, the pattern also implements
74
74
75
75
## Highlight: Using the CI Pipeline to provide supply chain security
76
76
77
-
This pattern includes some other technologies in the development CI pipeline, including cosign, a SIGSTORE project, implemented with [Tekton Chains](https://next.redhat.com/project/tekton-chains/). [Cosign](https://docs.sigstore.dev/cosign/overview/) supports container signing, verification, and storage in an OCI registry. It enables consumers to sign their pipeline resources and images and share the attestation files providing downstream consumers assurances that they are consuming a trusted artifact.
77
+
This pattern includes some other technologies in the development CI pipeline, including cosign, a SIGSTORE project, implemented with [Tekton Chains](https://next.redhat.com/2022/10/06/signing-images/). [Cosign](https://docs.sigstore.dev/cosign/overview/) supports container signing, verification, and storage in an OCI registry. It enables consumers to sign their pipeline resources and images and share the attestation files providing downstream consumers assurances that they are consuming a trusted artifact.
78
78
79
79
We also implement open source tools like [Sonarqube](https://www.sonarqube.org/) for static code analysis, [nexus](https://www.sonatype.com/products/nexus-repository?topnav=true) for securely storing build artifacts in-cluster, and an open source reports application that is used to upload and present the reports from the security pipeline.
Copy file name to clipboardExpand all lines: content/contribute/background-on-pattern-development.adoc
+29-26Lines changed: 29 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,18 +5,21 @@ weight: 20
5
5
aliases: /background-on-pattern-development/
6
6
---
7
7
8
-
# Background on pattern development
8
+
:toc:
9
+
:imagesdir: /images
10
+
:_content-type: ASSEMBLY
11
+
12
+
== Introduction
9
13
10
-
## Introduction
11
14
This section provides details on how to create a new pattern using the validated patterns framework. Creating a new pattern might start from scratch or it may start from an existing deployment that would benefit from a repeatable framework based on GitOps.
12
15
13
16
This introduction explains some of framework design decisions and why they were chosen. There are some high level concepts that are required for the framework. While those concepts can be implemented using a variety of open source projects, this framework is prescriptive and mentions the project and also (down stream) product that was used. E.g. For development builds we use Tekton (project) and specifically use OpenShift Pipelines (product).
14
17
15
-
The framework uses popular Cloud Native Computing Foundation (CNCF) [projects](https://landscape.cncf.io/) as much as possible. The CNCF landscape contains many projects that solve the same or similar problem. The validated patterns effort has chosen specific projects but it is not unreasonable for users to switch out one project for another. (See more on Operators below).
18
+
The framework uses popular Cloud Native Computing Foundation (CNCF) https://landscape.cncf.io/[projects] as much as possible. The CNCF landscape contains many projects that solve the same or similar problem. The validated patterns effort has chosen specific projects but it is not unreasonable for users to switch out one project for another. (See more on Operators below).
16
19
17
20
There is no desire to replicate efforts already in CNCF. If new a open source project comes out of this framework, the plan would be to contribute that to CNCF.
18
21
19
-
##Who is a pattern developer?
22
+
== Who is a pattern developer?
20
23
21
24
Many enterprise class Cloud Native applications are complex and require many different application services integrated together. Organizations can learn from each other on how to create robust, scalable, and maintainable systems. When you find a pattern that seems to work, it makes sense to promote best practices to others in order for them to not repeat the many failures you probably made while getting to your killer pattern.
22
25
@@ -26,77 +29,77 @@ This validated patterns framework has evolved since it was started in 2019. It w
26
29
27
30
Therefore, the goal is, that developers, operators, security, and architects will use this framework to have secure and repeatable day one deployment mechanism and maintenance automation for day two operations.
28
31
29
-
##A common platform
32
+
== A common platform
30
33
31
34
One of the most important goals of this framework is to provide consistency across any cloud provider - public or private. Public cloud providers each have Kubernetes distributions. While they keep up with the Kubernetes release cycle, they are not always running on the same version. Furthermore, each cloud provider has their own sets of services that developers often consume. So while you could automate the handling for each of the cloud providers, the framework utilizes one Kubernetes distribution that runs on public or private clouds - the hybrid and/or multi cloud model.
32
35
33
36
The framework depends on Red Hat OpenShift Container Platform (OCP). Once you have deployed Red Hat OCP wherever you wish to deploy your cloud native application pattern, then the framework can deploy on that platform in a few easy steps.
34
37
35
-
##Containment beyond containers
38
+
== Containment beyond containers
36
39
37
40
If you are reading this chances are you are already familiar with Linux containers. But there is more to containers than Linux containers in the Cloud Native environment.
38
41
39
-
###Containers
42
+
=== Containers
40
43
41
44
Containers allow you to encapsulate your program/process and all its dependencies in one package called a container image. The container runtime starts an instance of this container using only the Linux kernel and the directory structure, with program and dependencies, provided in the container image. This ensures that the program is running isolated from any other packages, programs, or files loaded on the host system.
42
45
43
46
Kubernetes, and the Cloud Native community of services, use Linux containers as their basic building block.
44
47
45
-
###Operators
48
+
=== Operators
46
49
47
50
While Linux containers provide an incredibly useful way to isolate the dependencies for an application or application service, containers also require some lifecycle management. For example, at start up a container my need to set up access to networks, or extra storage. This type of set up usually happens with a human operator deciding on how the container will connect networks or host storage. The operator may also have to do routine maintenance. For example, if the container contains a database, the human operator may need to do a backup or routine scrubbing of the database.
48
51
49
-
Kubernetes [Operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) are an extension to Kubernetes *"that make sue of custom resources to manage applications and their components."* I.e. it provides an extra layer of encapsulation on top of containers that packages up some operation automation with the container. It puts what the human operator would do into an Operator pattern for the service or set of services.
52
+
Kubernetes https://kubernetes.io/docs/concepts/extend-kubernetes/operator/[Operators] are an extension to Kubernetes _"that make sue of custom resources to manage applications and their components."_ I.e. it provides an extra layer of encapsulation on top of containers that packages up some operation automation with the container. It puts what the human operator would do into an Operator pattern for the service or set of services.
50
53
51
-
Many software providers/vendors have created operators to manage their application or service lifecycle. Red Hat OpenShift provides a [catalog of certified Operators](https://catalog.redhat.com/software/operators/search) that application develops can consume as part of their overall application. The validated patterns makes use of these certified Operators as much as possible. Having a common platform like Red Hat OpenShift helps reduce risk by using certified Operators.
54
+
Many software providers/vendors have created operators to manage their application or service lifecycle. Red Hat OpenShift provides a https://catalog.redhat.com/software/operators/search[catalog of certified Operators] that application develops can consume as part of their overall application. The validated patterns makes use of these certified Operators as much as possible. Having a common platform like Red Hat OpenShift helps reduce risk by using certified Operators.
52
55
53
-
###Validated patterns
56
+
=== Validated patterns
54
57
55
58
Assembling operators into a common pattern provides another layer of encapsulation. As with an Operator, where the developer can take advantage of the best practices from a experienced human operator, a validated pattern provides a way of taking advantage of best practices for deploying operators and other assets for a particular type of solution. Rather than starting from scratch to figure out how to deploy and manage a complex set of integrated and dependent containerized services, a developer can take a validated pattern and know that a lot of experience has been put into it.
A validated pattern has been tested and continues to be tested as the lifecycle of individual parts (Operators) change through release cycles. Red Hat's Quality Engineering team provides Continuous Integration of the pattern for new releases of Red Hat products (Operators).
60
63
61
64
The validated patterns framework takes advantage of automation technology. It uses Cloud Native automation technology as much as possible. Occasionally the framework resorts to some scripts in order to get a pattern up and running faster.
62
65
63
-
##Automation has many layers
66
+
== Automation has many layers
64
67
65
68
As mentioned above, gaining consistency and robustness for deploying complex Cloud Native applications requires automation. While many Kubernetes distributions, including OpenShift, provide excellent user interfaces for deploying and managing applications, this is mostly useful during development and/or debugging when things go wrong. Being able to consistently deploy complex applications is critical.
66
69
67
70
But which automation tool should be used? Or which automation tools, plural? During the development of the validated patterns framework we learned important lessons on the different areas of automation.
68
71
69
-
###Automation for building application code
72
+
=== Automation for building application code
70
73
71
-
When developing container based Cloud Native applications, a developer needs to build executable code and create a new container image for deployment into their Kubernetes test environment. Once tested, that container image needs to be moved through the continuous integration and continuous deployment (CI/CD) pipeline until it ends up in production. [Tekton](https://tekton.dev/) is a Cloud Native CI/CD project that is build for hybrid-cloud. [OpenShift Pipelines](https://cloud.redhat.com/learn/topics/ci-cd) is a Red Hat product based on Tekton.
74
+
When developing container based Cloud Native applications, a developer needs to build executable code and create a new container image for deployment into their Kubernetes test environment. Once tested, that container image needs to be moved through the continuous integration and continuous deployment (CI/CD) pipeline until it ends up in production. https://tekton.dev/[Tekton] is a Cloud Native CI/CD project that is build for hybrid-cloud. https://cloud.redhat.com/learn/topics/ci-cd[OpenShift Pipelines] is a Red Hat product based on Tekton.
72
75
73
-
###Automation for application operations
76
+
=== Automation for application operations
74
77
75
-
There are two aspects to consider for operations when doing automation. First, you must be able to package up much of the configuration that is required for deploying Operators and pods. The validated patterns framework started with a project called Kustomize which allows you to assemble complex deployment YAML to apply to your Kubernetes cluster. Kustomize is a powerful tool, and almost achieved what we needed. However it fell short when we needed to propagate variable data into our deployment YAML. Instead we chose [Helm](https://cloud.redhat.com/learn/topics/helm) because it provides templating and can therefore handle the injection of variable data into the deployment package. See more on templating [here](https://helm.sh/docs/helm/helm_template/).
78
+
There are two aspects to consider for operations when doing automation. First, you must be able to package up much of the configuration that is required for deploying Operators and pods. The validated patterns framework started with a project called Kustomize which allows you to assemble complex deployment YAML to apply to your Kubernetes cluster. Kustomize is a powerful tool, and almost achieved what we needed. However it fell short when we needed to propagate variable data into our deployment YAML. Instead we chose https://cloud.redhat.com/learn/topics/helm[Helm] because it provides templating and can therefore handle the injection of variable data into the deployment package. See more on templating https://helm.sh/docs/helm/helm_template/[here].
76
79
77
80
The second aspect of automation for application automation deals with both workflow and GitOps. Validated patterns requires that a workflow deploys various components of the complex application. Visibility into the success or failure of those application components is really important. After the initial deployment it is important to role out configuration changes in an automated way using a code repository. This is achieved using GitOps. I.e. Using a Git repository as a mechanism to change configuration that triggers the automatic roll-out of those changes.
78
81
79
-
*"Application definitions, configurations, and environments should be declarative and version controlled. Application deployment and lifecycle management should be automated, auditable, and easy to understand."* - Argo CD project
82
+
_"Application definitions, configurations, and environments should be declarative and version controlled. Application deployment and lifecycle management should be automated, auditable, and easy to understand."_ - Argo CD project
80
83
81
-
OpenShift GitOps is based on the [Argo CD](https://argo-cd.readthedocs.io/en/stable/) project. It is a GitOps continuous delivery tool for Kubernetes.
84
+
OpenShift GitOps is based on the https://argo-cd.readthedocs.io/en/stable/[Argo CD] project. It is a GitOps continuous delivery tool for Kubernetes.
82
85
83
-
##Secret handling
86
+
== Secret handling
84
87
85
88
Validated patterns often depend on resources that require certificates or keys. These secrets need to be handled carefully. While it's tempting to focus on just the deployment of a pattern and "handle security later", that's a bad idea. In the spirit of DevSecOps, the validated patterns effort has decided to "shift security left". I.e. build security in early in the lifecycle.
86
89
87
90
When it comes to security, the approach requires patience and care to set up. There is no avoiding some manual steps but validated patterns tries to automate as much as possible while at the same time taking the lid off so developers can see what was and needs to be done.
88
91
89
92
There are two approaches to secret handling with validated patterns:
90
93
91
-
- Using special configuration files. This is fine for initial development but not for production.
92
-
- Using a Cloud Native secrets handling tool e.g. [Vault](https://www.vaultproject.io/docs/platform/k8s) or [Conjur](https://www.conjur.org/solutions/secrets-management/)
94
+
* Using special configuration files. This is fine for initial development but not for production.
95
+
* Using a Cloud Native secrets handling tool e.g. https://www.vaultproject.io/docs/platform/k8s[Vault] or https://www.conjur.org/solutions/secrets-management/[Conjur]
93
96
94
-
Some of the validated patterns use configuration files (for now), while others, like the [Multicloud GitOps](https://validatedpatterns.io/multicloud-gitops/), use Vault. See [Vault Setup](https://validatedpatterns.io/secrets/vault/) for more info.
97
+
Some of the validated patterns use configuration files (for now), while others, like the https://validatedpatterns.io/multicloud-gitops/[Multicloud GitOps], use Vault. See https://validatedpatterns.io/secrets/vault/[Vault Setup] for more info.
95
98
96
-
##Policy
99
+
== Policy
97
100
98
101
While many enterprise Cloud Native applications are open source, many of the products used require licenses or subscriptions. Policies help enforce license and subscription management and the channels needed to get access to those licenses or subscriptions.
99
102
100
-
Similarly, in multicloud deployments and complex edge deployments, policies can help define and select the correct GitOps workflows that need to be managed for various sites or clusters. E.g. defining an OpenShift Cluster as a "Factory" in the [Industrial Edge](https://validatedpatterns.io/industrial-edge/factory/) validated pattern provides a simple trigger to roll-out the entire Factory deployment. Policy is a powerful tool in automation.
103
+
Similarly, in multicloud deployments and complex edge deployments, policies can help define and select the correct GitOps workflows that need to be managed for various sites or clusters. E.g. defining an OpenShift Cluster as a "Factory" in the https://validatedpatterns.io/industrial-edge/factory/[Industrial Edge] validated pattern provides a simple trigger to roll-out the entire Factory deployment. Policy is a powerful tool in automation.
101
104
102
-
Validated patterns use [Red Hat Advanced Cluster Management for Kubernetes](https://www.redhat.com/en/technologies/management/advanced-cluster-management) to control clusters and applications from a single console, with built-in security policies.
105
+
Validated patterns use https://www.redhat.com/en/technologies/management/advanced-cluster-management[Red Hat Advanced Cluster Management for Kubernetes] to control clusters and applications from a single console, with built-in security policies.
0 commit comments