What is CircleCI?

CircleCI is a platform for continuous integration and continuous deployment (CI/CD). Established in 2011 and valued at $1.7 billion in 2021, it stands as one of the most favored CI/CD platforms globally. CircleCI empowers developers to build, test, and deploy their code swiftly and reliably. Today, the company supports thousands of organizations around the world, from startups to major corporations, providing the flexibility and power needed for any project type.

But what exactly is CI/CD?

CI/CD, or “Continuous Integration / Continuous Delivery“, is a development methodology focusing on automating the steps within an application’s lifecycle. It aims to decrease human errors, speed up delivery times, and enhance application quality.

• Continuous Integration (CI) involves regularly incorporating code written by various team members into a central repository. Each modification is automatically tested to ensure it doesn’t introduce bugs or compatibility issues.
• Continuous Deployment (CD) automates the delivery of validated code without requiring manual intervention.

Key Features

CircleCI offers a comprehensive suite of tools and integrations aimed at automating, optimizing, and securing the development process while ensuring efficient continuous delivery.

Flexible Workflows

Facilitates orchestration of complex pipelines with conditional dependencies, optimizing tests and deployments to save time.

Cloud Execution

Delivers scalable cloud environments using virtual machines or Docker.

Build Parallelization

Distributes tasks to hasten tests and builds, reducing processing time.

Multi-environment Support

Supports a variety of languages, frameworks, and environments.

Deployment Automation

Integrates with tools like Kubernetes and Terraform.

Intelligent Caching

Caches dependencies to avoid re-downloading them.

Security and Compliance

Features include data encryption, granular permissions, and pipeline audits.

Continuous Testing and Reporting

Integrates with testing tools to generate detailed reports.

Monitoring and Insights

Analytical dashboards for monitoring pipeline performance.

Git Integration

Works seamlessly with GitHub, GitLab, and Bitbucket to automate CI/CD workflows.

Setting Up CircleCI

The initial setup is straightforward and quick, supporting integration with GitHub, GitLab, or Bitbucket.

1. Before Installation

• Initially, ensure you have an account on a compatible code management platform (GitHub, GitLab, or Bitbucket) and a CircleCI account linked to the repository in use.
• A configuration file config.yml must be in a .circleci directory at the repository root.

2. Project Registration

On the CircleCI website, grant the necessary permissions for it to access your repositories. To add a new project, click on “Set up project” as shown in the screenshot below: Select an integration method; in this example, “Fast” is chosen. A configuration file is generated automatically. Learn how to run a CircleCI project

3. The Configuration File

In our example, the resulting configuration file is:

version: 2.1
orbs:
  python: circleci/[email protected]
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.12
    steps:
      - checkout
      - python/install-packages:
          pkg-manager: pip
      - run:
          name: Run tests
          command: pytest
workflows:
  sample: 
    jobs:
      - build-and-test

• The first line indicates the schema version used by CircleCI.
• Orbs are reusable packages containing preconfigured CircleCI configurations for rapid integration of specific tools or languages without rewriting the setup.
• Jobs are work units comprising sequential steps, referred to as steps, executed in a defined environment. In this context, the build-and-test task includes a docker element defining the execution environment (a Docker image provided by CircleCI), along with the steps described below.
• Steps (steps) outline actions performed within a job:
  • checkout: Retrieves the source code from the repository, making it accessible within the CircleCI environment.
  • python/install-packages: Installs necessary dependencies through the pip package manager.
  • run: Executes a job command. Here, Pytest is utilized to perform unit tests.
• Lastly, workflows are collections of jobs executed in a predetermined sequence:
  • sample: Name of the workflow
  • jobs: List of jobs to execute in the workflow

Potential modifications could include:

• To avoid reinstalling packages with every run (thus saving build time), implement cache management:

- restore_cache:
    keys:
      - v1-dependencies-{{ checksum "requirements.txt" }}
- python/install-packages:
    pkg-manager: pip
- save_cache:
    key: v1-dependencies-{{ checksum "requirements.txt" }}
    paths:
      - ~/.cache/pip

How to configure CircleCI This enables CircleCI to attempt restoring an existing cache based on the requirements.txt file hash. If found, they’ll be retrieved; if not, they’ll be installed via pip.

• Upon moving the application to production, automated deployments can be added following successful tests:

workflows:
  version: 2
  deploy:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
          filters:
            branches:
              only: main

4. Cloud Integration

CircleCI facilitates seamless integration with cloud infrastructures, including Azure, AWS, or GCP. In this instance, we perform an AWS integration. To simplify the process, add an orb as follows:

aws-cli: circleci/[email protected]

Also, incorporate a job resembling the example below:

deploy-to-aws:
    docker:
      - image: cimg/python:3.12
    steps:
      - attach_workspace: 
          at: /home/circleci/project
      - aws-cli/setup
      - run:
          name: Configure AWS credentials
          command: |
            aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
            aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
            aws configure set default.region $AWS_DEFAULT_REGION
      - run:
          name: Deploy application to S3
          command: |
            aws s3 cp /home/circleci/project/ s3://$AWS_S3_BUCKET_NAME --recursive
      - run:
          name: Clear AWS credentials (for security)
          command: rm -f ~/.aws/credentials

The environment variables (identifiable by the $ symbol) should be pre-created in CircleCI settings. Mastering cloud integration on CircleCI

CI/CD Best Practices

CircleCI suggests several optimal practices for enhancing CI/CD pipelines. The CI/CD process is deemed essential for every development team. Relying on manual processes impedes development speed. Reuse configurations and devise caching strategies. Avoid duplicating configurations and redundantly downloading dependencies for every build. Leverage CircleCI orbs to automate repetitive tasks. Refrain from manually scripting redundant configurations for every project. Secure pipelines with private environment variables, varied contexts, and administrative oversight. Avoid storing sensitive data in project configuration files. Ensure comprehensive test coverage by integrating partners like Codecov, Cypress, or SonarCloud. Never deploy code without testing or confirming test coverage. Utilize automatic Slack notifications to monitor pipeline status and swiftly address failures. Avoid overlooking notifications, as they could lead to delays in issue detection. Incorporate popular orbs like Node, AWS-CLI, and Python to streamline configurations and cloud deployment. Avoid manually configuring each tool, which may result in errors and inconsistencies. Examine pipeline metrics to pinpoint bottlenecks and enhance workflow efficiency. Neglecting pipeline performance monitoring can hinder deployment speed.

Conclusion

CircleCI is a powerful and adaptable solution that enables the automation and optimization of CI/CD practices. Training with Liora