12 factor apps

12-factor apps

#1. One codebase, 1 application

• Code base to be backed by a version control system such as Git, Subversion etc., 
• 1 code base = 1 app. 
• Multiple apps sharing the same code is a violation of the 12-factor. 

#2. Dependencies

Manage dependencies in your application manifest. 

• Database, image processing libraries. 

Characteristics:

• Dependencies are managed in the app manifests such as maven, gradle etc., 

What should you do or don’t do: 

• Don’t use pre-installed softwares which will help in each environments. This will not automate the deployment. 
• Don’t assume that the related dependencies will be in the environment where you deploy, you are responsible for wiring the dependencies. 

#3. Externalize configuration

Application configuration referred here are values that are

• Credentials to access a database, or services such as S3. 
• Environment specific properties

Motivation: 

• Env properties stored in the code will be a violation since the code has to be redeployed whenever the properties changes. 
• Also its not a good idea to store env values in the code.

#4. Backing services

Are services that the app talks to 

Characteristics

• Services can be attached or detached whenever required. 

Objective:

• A different SMTP server should be able to be configured without the need for code change. 

#5. Build release run

Summary: 

Build: Application code gets converted into an artifact such as a war file. 

Release: the artifact now understands the environment - QA, Dev, Prod

Run: the app gets released in the specific environment. 

Characteristics

• Strict separation between the build, release and run stages of the application. 
• Every release must have a release id such as a timestamp or a incrementing number
• 1-click release - using CI/CD tool. 

What should you do: 

• Use a CI/CD tool, Jenkins, Concourse etc.,

Best Practices:

• Create smoke tests to ensure the app is running fine after deployment. 

#6. Stateless Processes

Characteristics:

• The process running in the environment should be stateless. 
• Should not share anything with other process. 
• No sticky sessions

What should do or don’t do. 

• Don’t store any data in a local file system. 
• Sessions: 
• Store your sessions if any in a distributed session storage db, such as Redis. 
• No sticky sessions. 
• Create stateless services. 

#7. Port binding

    The port binding for the application should be configurable and should not depend upon a particular infrastructure setup. 

#8. Concurrency

#9. Disposability

Your application should work fine even though one or more of app instances die. 

#10. Dev/Prod parity

Keep the developer environment as close as possible to the production environment. 

#11. Logs

Ensure it is easy to view / debug logs even though there may be multiple instances of the services exists. 

What should you do or don’t do:

• No System.out.printlns
• Write to log files on the web server using log4j like frameworks. 
• Stream logs to an external server where it can be viewed easily. Send the logs to a centralized logging facility. 

#12. Admin processes

Characteristics: 

• Execute any migration scripts on deployment of a startup if any. 

What should you do or don’t do: 

• Store migration scripts in the repository. 

Docker - getting started (very basics)

Docker-Overview

Docker makes much easier to deploy applications. It helps to resolve one main problem - “works in my machine problem”. 

Getting started

• Download docker from here for your OS: 

• https://www.docker.com/community-edition#/download

• Install the docker DMG if Mac.

• Login to docker once it is installed. 

• Open terminal and execute the following: 
• Docker -version
• Docker run hello-world
• Docker run —name deiveehello hello-world
• Docker run -it —name mylinuc-container ubuntu bash

For a hands on video - click here

——————————————————————

Some basic docker commands

——————————————————————

Docker images // lists all images

Docker ps // lists all process

Docker run —help

———————————————————————

Running a docker using simple ubuntu image

———————————————————————

docker ps -a -f status=exited -q

docker rm $(docker ps -a -f status=exited -q)

docker run -it --name my-linuc-container-1 ubuntu bash

docker run -it --name my-linux-container1 --rm -v /Users/deiveehannallazhagappan/Documents/volumes/project1:/my-data ubuntu bash

Cd my-data

Touch aa.txt

Ls

———————————————————————

Creating image from a local docker file.

———————————————————————

• Create a docker file with the following content.

FROM ubuntu

CMD echo "Hello Deivee"

• Run the following command. 

docker $docker build -t deivee-tracker-image .

Sending build context to Docker daemon  2.048kB

Step 1/2 : FROM ubuntu

 ---> f975c5035748

Step 2/2 : CMD echo "Hello Deivee"

 ---> Running in 10c68c4eb454

Removing intermediate container 10c68c4eb454

 ---> b235e94e39b4

Successfully built b235e94e39b4

Successfully tagged deivee-tracker-image:latest

docker $docker images

REPOSITORY             TAG                 IMAGE ID            CREATED              SIZE

deivee-tracker-image   latest              b235e94e39b4        About a minute ago   112MB

ubuntu                 latest              f975c5035748        4 weeks ago          112MB

hello-world            latest              f2a91732366c        4 months ago         1.85kB

———————————————————————

Run the docker image

———————————————————————

docker $docker run deivee-tracker-image

Hello Deivee

docker $docker ps -a

CONTAINER ID        IMAGE                  COMMAND                   CREATED             STATUS                      PORTS               NAMES

9a65c141ec46        deivee-tracker-image   "/bin/sh -c 'echo \"H…"   5 seconds ago       Exited (0) 4 seconds ago                        optimistic_stonebraker

0e922f16f866        ubuntu                 "bash"                    12 minutes ago      Exited (0) 11 minutes ago                       tracker

be1311055d15        ubuntu                 "bash"                    23 minutes ago      Created                                         my-linux-container

77447d4d6c02        hello-world            "--name deiveehello"      12 hours ago        Created                                         elastic_wozniak

For a more hands on video on getting started with Docker, click here.
https://www.youtube.com/watch?v=hfL0USkCmZ0&t=305s

Java 8 - new important features

Java 8 has included a lot of new features, this post will explain a few important features. 

1. Lamda Expressions. 
2. Functional Interfaces. 
3. Streams
4. Nashorn

Other features. 

• Date and time api changes. 
• Foreach. 

——————————————

Lamda expressions

——————————————

Lamda expression is a function with no name and with implementation. 

A basic expression format would be..

(x, y) -> x + y

Where (x, y) are the function input parameters. And x+y is the statement executed by the function and returned. 

Example: 

FileFilter filterLamda = (File pathName) -> pathName.getName().endsWith(".png");

File dir = new File("/Users/deiveehannallazhagappan/workspace/xplore-java/supporting");

File[] files = dir.listFiles(filterLamda);

for (File f: files) {

    System.out.println("f = " + f);

}

——————————————

Functional Interfaces

——————————————

Functional interfaces are interfaces that have only one abstract method in it. 

Example: 

@FunctionalInterface

public interface TestFuncInterface {

    public void testmethod();

default void testdefaultmethod(){

         System.out.println(“default method“);

     }

}

Lamda expressions can be used to represent the instance of a functional interface. 

Note: 

• You can add default methods .

——————————————

Streams

——————————————

Streams are new feature introduced in Java 8 to simplify the logic required to process data - filtering, transforming data etc., 

For example, the below example does the following

• Creates a new array list from 1-30
• Creates a stream of integers. 
• Filters the list by extracting only no.s divisible by 3 and converts them into a integer array. 

Stream example: 

List<Integer> list = new ArrayList<Integer>();

for(int i = 1; i< 30; i++){

    list.add(i);

}

Stream<Integer> stream = list.stream();

Integer[] evenNumbersArr = stream.filter(i -> i%3 == 0).toArray(Integer[]::new);

System.out.print(evenNumbersArr);

Why streams: reduce complexity, less code. 

——————————————

Nashorn

——————————————

Nashorn is a javascript engine which allows the java applications to interact with the JS code. 

ScriptEngineManager sem = new ScriptEngineManager();

ScriptEngine engine = sem.getEngineByName("nashorn");

engine.eval("print('deiveehan')");

You can also execute an external script instead of using inline js code. 

engine.eval(new FileReader("/Users/deiveehannallazhagappan/workspace/xplore-java/supporting/js/deiveescript.js"));

You will get the output of the js code when you run the java application. 

Refreshing - Git basics

This post explains some of the basics of git (a simple refresher post to GIT commands).

Git clone <git url> download a project from git remote repository

init

add <file> or .

commit -m <commit message> (Commit files to local repository)

status (list all new and modified files)

diff (Shows files modified)

fetch: fetches the remote repo into the local repo, but does not merge

pull: does and fetch and also does the merge into the local repo. 

Remote: lists the remote

git remote

git remote -v

Log: logs all changes

git log

git log —all

git log —all —oneline —decorate —graph

SCENARIOS:

———————————————————

# Local folder to Git remote. 

————————————————————

Steps to commit an already existing local project to git. 

git init

git add README.md

Git commit -m “Initial commit”

Create a repository in git and get the 

git remote add origin git@github.com:deiveehan/<reponame.git>

Git push -u origin master. 

————————————————————

# Create a branch from command line.

————————————————————

git branch —list (shows all the branches). 

git checkout -b develop (creates a local branch called develop)

change a file

perform git add and commit to the local code.

git push —set-upstream origin develop

————————————————————

# Switching branches.

————————————————————

Git branch (shows local branches)

Git checkout develop

————————————————————

# Merging with parent branch (say develop)

————————————————————

Master

• Develop
• A

commit A changes to local repository

git checkout develop

git merge A

git push

————————————————————

# Deleting remote branch

————————————————————

git branch -d A (Delete references of local branch)

git push origin -d A (delete remote branch)

————————————————————

Garbage collection

Terms:

• Young generation (Minor GC). 
• Eden: 
• New objects gets allocated here. 
• From survivor
• Once eden space is full, minor gb occurs and moves from from to to survivor. 
• To Survivor
• Old generation (Tenured) (Major GC) 
• Oldest objects to old generation
• Perm generation 

Note: 

• SerialGen operates on the young gen. 
• Concurrent gen operates on the old gen. 

-XX:+PrintClmmandLineFlags -version TestSystemGC

Concepts: 

Minor GC:

Garbage collection algorithms,

• MinorJS: 
• Serial Copy collector:  -XX:+UseSerialGC
• -XX:+UseSerialGC
• When to use: 
• Parallell Scavenge Collector: -XX+UseParallelGC
• Parallel copy collector: -XX:+UseParNewGC
• Garbage first collector: -XX:+UseG1GC
• Major JC:

• Marksweep compact collector: -XX:+UseSerialGC
• Parallell scavenge marksweep collector: -XX:+UseConcMarkSweepGC
• Gargbage first collector: -XX:+UseG1GC

Full GC:

• When full GC occurs, the Full GC pause time is normally lengthier. 

What you need to know

• Frequency in which minor GC. 
• Object allocation rate
• Size of the eden space. 
• Frequency of object promotion to old generation
• Frequency of minor GC (how quickly objects age)
• Size of survivor space. 

Steps: 

• Check the frequency of Minor GC: (Defined by allocation rate and size of Eden)
• Higher allocation rate and smaller eden space: More frequent minor GC
• Lower allocation rate or larger eden space: less frequent minor GC. 
• Check the frequency of Full GC: (Defined by promotion rate and size of old generation space)
• For Parallel GC 
• Higher promotion rate and small old generation space: frequent full GC
• Lower promotion rate and larger old men: less frequent full GC.
• For CMS & G1
• G1: 
• Avoids fragmentation. 
• CMS: 

Notes:

• The longer the object lives, the greater the impact on throughput latency and footprint
• Object retention can degrade performance more than object allocation. 
• GC visits only visits live objects. GCs love small immutable objects. 
• Object allocation is very cheap (10 CPU instructions). 

Takeaways:

• Its better to use short lived immutable objects vs long lived mutable objects. 
• Start with -XX:+UseParallelOldGC and avoid full GC. 
• If there are frequent full GC then move to CMS or G1 if needed (for old gen collections). 

Best practices: 

• Avoid data structure resizing. 
• Avoid large allocation. 
• Dont do finalizers (requires GC cycles and GC cycles are slower), if required use reference objects as an alternatives. 
• Soft references (don’t do it). 

12-factor apps

#1. One codebase, 1 application

• Code base to be backed by a version control system such as Git, Subversion etc., 
• 1 code base = 1 app. 
• Multiple apps sharing the same code is a violation of the 12-factor. 

#2. Dependencies

Manage dependencies in your application manifest. 

• Database, image processing libraries. 

Characteristics:

• Dependencies are managed in the app manifests such as maven, gradle etc., 

What should you do or don’t do: 

• Don’t use pre-installed softwares which will help in each environments. This will not automate the deployment. 
• Don’t assume that the related dependencies will be in the environment where you deploy, you are responsible for wiring the dependencies. 

#3. Externalize configuration

Application configuration referred here are values that are

• Credentials to access a database, or services such as S3. 
• Environment specific properties

Motivation: 

• Env properties stored in the code will be a violation since the code has to be redeployed whenever the properties changes. 
• Also its not a good idea to store env values in the code.

#4. Backing services

Are services that the app talks to 

Characteristics

• Services can be attached or detached whenever required. 

Objective:

• A different SMTP server should be able to be configured without the need for code change. 

#5. Build release run

Summary: 

Build: Application code gets converted into an artifact such as a war file. 

Release: the artifact now understands the environment - QA, Dev, Prod

Run: the app gets released in the specific environment. 

Characteristics

• Strict separation between the build, release and run stages of the application. 
• Every release must have a release id such as a timestamp or a incrementing number
• 1-click release - using CI/CD tool. 

What should you do: 

• Use a CI/CD tool, Jenkins, Concourse etc.,

Best Practices:

• Create smoke tests to ensure the app is running fine after deployment. 

#6. Stateless Processes

Characteristics:

• The process running in the environment should be stateless. 
• Should not share anything with other process. 
• No sticky sessions

What should do or don’t do. 

• Don’t store any data in a local file system. 
• Sessions: 
• Store your sessions if any in a distributed session storage db, such as Redis. 
• No sticky sessions. 
• Create stateless services. 

#7. Port binding

#8. Concurrency

#9. Disposability

Your application should work fine even though one or more of app instances die. 

#10. Dev/Prod parity

Keep the developer environment as close as possible to the production environment. 

#11. Logs

Ensure it is easy to view / debug logs even though there may be multiple instances of the services exists. 

What should you do or don’t do:

• No System.out.printlns
• Write to log files on the web server using log4j like frameworks. 
• Stream logs to an external server where it can be viewed easily. Send the logs to a centralized logging facility. 

#12. Admin processes

Characteristics: 

• Execute any migration scripts on deployment of a startup if any. 

What should you do or don’t do: 

• Store migration scripts in the repository. 

Externalizing configurations.

Motivation why to externalize configuration:

• Externalize config data from the applications across all environments. 
• Store crucial information such as password for DB in a centralized place. 
• Share data between different services. 
• See who changed which property and when and why. 
• Build once deploy anywhere. 

What is Cloud Config server: 

Its a way by which we can externalize the property values by moving out of the project. Primarily kept for properties that you normally override or keep secret values out of the project src. 

What should you do to use Spring cloud config

1. Application configuration: 
• Manage app configuration in a git hub or a local file system. 
2. Config server
• Point the location of GIT or local file system where the app config is located (i.e., GIT URI)
• @EnableConfigServer
3. Config client
• Use it normally as how you would read from the properties file 
• @Value or 

Features

• Many storage options including Git, Subversion. 
• Pull model
• Traceability
• @RefreshScope for allowing properties to refresh. Calls the constructor of the beans and reinitializes it. 
• @ConfigurationProperties alternate for @RefreshScope which does reinitialize the Bean that is annotated with @ConfigurationProperties. 
• Encrypt and decrypt property values. 

Usecases for Cloud config: 

• Change log level in the properties. 
• Environment config - containing passwords. 
• Toggle a feature on or off. 
• Using @RefreshScope
• @ConfigurationProperties. 

Order in which the Cloud config looks into the properties files. 

• Config server. 
• Command line parameters
• System properties.
• Classpath: application.yml file
• Classpath: bootstrap.yml

Things that you can try: 

• Simple example to demonstrate properties in Git and reading from config server and client 
• Using maven profiles specific properties in Config server in Github. 
• Managing log levels in the Config server
• Using @RefreshScope and @CofigurationProperties. 

Best practices: 

• Config server communication with the Git hub, pull every time when client requests for properties file. Use a Jenkins job which will push the properties to a Config server properties (clone). 
• Single repository for all teams, subdirectories for different environments. 
• Source code maintenance, having branch per release vs always using the master. 
• Managing encrypted values for secure data, using JWT with Config server.
• Use Spring cloud bus to refresh all the services which listens to the rabbitmq.

Note: 

• Store only things that need to be externalized, I.e., Resource bundles for internationalization should be at the project level.