Log drain (log management service - Logit)

CF aggregates the logs from all the instances and make them available whenever you say cf logs <app name>

This displays logs from all the instances in the app. 

However this only shows the recent logs and if you want to view more logs, then you have to drain the logs to a log management services

Steps to stream logs to a log management service: 

1: Log service configuration. 

a. Go to the third party log service provider, signup. 

https://logit.io/

d. Get the syslog url provided by the third party service. 

syslog-tls://d0cde9b8-8275-41f8-bccf-af38173c62e7-ls.logit.io:21298

2: Create a User provided service instance. 

a. create CUPS -l <syslog url>

$cf cups logit-ssl-drain -l syslog-tls://d0cde9b8-8275-41f8-bccf-af38173c62e7-ls.logit.io:21298

b. Bind the service instance to the app that needs logging. 

$cf bind-service userservice logit-ssl-drain

3: Verify the logs

Go to your service provider and view the logs. 

Managing service instances in PCF

We will look at how to we use the services available in the market place. 

Some basic commands: 

cf marketplace : list all services in the marketplace

cf services: list all service instances created. 

Creating and binding a service from marketplace. 

View list of all the services in the market place

/ $cf marketplace

Getting services from marketplace in org pcfdev-org / space pcfdev-space as user...

OK

service        plans             description

local-volume   free-local-disk   Local service docs: https://github.com/cloudfoundry-incubator/local-volume-release/

p-mysql        512mb, 1gb        MySQL databases on demand

p-rabbitmq     standard          RabbitMQ is a robust and scalable high-performance multi-protocol messaging broker.

p-redis        shared-vm         Redis service to provide a key-value store

Create a new service instance

/ $cf create-service p-mysql 512mb user-db

Creating service instance user-db in org pcfdev-org / space pcfdev-space as user...

/ $cf apps

Getting apps in org pcfdev-org / space pcfdev-space as user...

OK

name          requested state   instances   memory   disk   urls

userservice   started           2/2         512M     512M   userservice-dev.local.pcfdev.io

/ $cf services

Getting services in org pcfdev-org / space pcfdev-space as user...

OK

name      service   plan    bound apps   last operation

user-db   p-mysql   512mb                create succeeded

/ $cf bind-service userservice user-db

Binding service user-db to app userservice in org pcfdev-org / space pcfdev-space as user...

OK

Unbinding and deleting services

/ $cf services

Getting services in org pcfdev-org / space pcfdev-space as user...

OK

name      service   plan    bound apps    last operation

user-db   p-mysql   512mb   userservice   create succeeded

/ $cf unbind-service userservice user-db

Unbinding app userservice from service user-db in org pcfdev-org / space pcfdev-space as user...

OK

/ $cf delete-service user-db

Really delete the service user-db?> y

Deleting service user-db in org pcfdev-org / space pcfdev-space as user...

OK

/ $cf services

Getting services in org pcfdev-org / space pcfdev-space as user...

OK

No services found

/ $

list information about a specific service. 

Cf service user-db

You may not want to bind the service everytime you push an application. You can use the following entries to bind in the app manifest file. 

services:

• user-db

Some services require to pass some json attributes while binding the services. 

 cf bind-service userservice user-db-c ‘{“ref-name”:”DeiveeService”, role=“Admin”}’

You can also send as a json file. 

Cf bind-service userservice user-db -c /tmp/configuration.json

Other commands 

cf rename-service user-db myuser-db

cf update-service user-db -p 1GB

SSH ing into PCF application and service instances

Developers normally dont have to SSH into the pcf application or services. they normally view logs using cf logs. However if you really have to ssh into instances to troubleshoot something, then here are the steps. 

First you should have access to SSH into the application. Typically this depends upon your role. you can use the following commands to enable or disable ssh. ssh access can be provided at the app level or space level. 

Please find the below basic commands for enabling or disabling ssh access. 

SSHing directly using CF command

cf enable-ssh <app name>

cf disable-ssh <app name>

cf allow-space-ssh <space name>

cf disallow-space-ssh <space name>

cf ssh-enabled userservice

cf space-ssh-allowed pcfdev-space

Now that you have access to SSH, we can see the following

1. SSH into applications

2. SSH into service instances (such as database services )

1. —— SSH ing into apps ——

There are 2 ways to SSH one using CF CLI and other not using CF CLI (basically other ways). 

Accessing SSH without CF CLI

cf ssh <app name> -I <instance number to login> (optional)

$ cf ssh userservice

vcap@6fe377f0-43cd-4631-641f-c87ac239069e:~$ ls

app  logs  staging_info.yml  tmp

Get the GUID of the application. 

$ cf app userservice --guid

355aa3f0-2d28-4c52-a537-980e9d02f97e

Get the environmental information. 

$ cf curl /v2/info

{

   "name": "pcfdev",

   "build": "037c3c92eaf4bcab911fb12f09b0cdd464c0a9b2",

   "support": "pcfdev@pivotal.io",

   "version": 0,

   "description": "",

   "authorization_endpoint": "https://login.local.pcfdev.io",

   "token_endpoint": "https://uaa.local.pcfdev.io",

   "min_cli_version": null,

   "min_recommended_cli_version": null,

   "api_version": "2.65.0",

   "app_ssh_endpoint": "ssh.local.pcfdev.io:2222",

   "app_ssh_host_key_fingerprint": "a6:d1:08:0b:b0:cb:9b:5f:c4:ba:44:2a:97:26:19:8a",

   "app_ssh_oauth_client": "ssh-proxy",

   "routing_endpoint": "https://api.local.pcfdev.io/routing",

   "logging_endpoint": "wss://loggregator.local.pcfdev.io:443",

   "doppler_logging_endpoint": "wss://doppler.local.pcfdev.io:443",

   "user": "89bbff6f-bda1-4efe-b38a-2cc6f020217b"

}

Get the ssh-code

$ cf ssh-code

DhqVXotyRA

SSH into the instance provide the guid, and serverinformation, also provide the ssh-code obtained as the password. 

$ ssh -p 2222 cf:355aa3f0-2d28-4c52-a537-980e9d02f97e/0@ssh.local.pcfdev.io

cf:355aa3f0-2d28-4c52-a537-980@ssh.local.pcfdev.io's password: 

vcap@6fe377f0-43cd-4631-641f-c87ac239069e:~$ 

Now that we know how to ssh into the application, we can have a look at ssh ing into the services. 

—— SSH ing into services ——

View the services

$ cf services

Getting services in org pcfdev-org / space pcfdev-space as user...

OK

name      service   plan    bound apps    last operation

user-db   p-mysql   512mb   userservice   create succeeded

Create a service key, mention the service instance id and a service key. 

$ cf create-service-key user-db EXTERNAL-ACCESS-KEY

Creating service key EXTERNAL-ACCESS-KEY for service instance user-db as user...

OK

Get the service details using the key. 

$ cf service-key user-db EXTERNAL-ACCESS-KEY

Getting key EXTERNAL-ACCESS-KEY for service instance user-db as user...

{

 "hostname": "mysql-broker.local.pcfdev.io",

 "jdbcUrl": "jdbc:mysql://mysql-broker.local.pcfdev.io:3306/cf_05a179d8_9b8f_470f_9b81_14e4b4877c8b?user=FfbDMZl571mskftA\u0026password=MyfJQwpcq82lyJfh",

 "name": "cf_05a179d8_9b8f_470f_9b81_14e4b4877c8b",

 "password": "MyfJQwpcq82lyJfh",

 "port": 3306,

 "uri": "mysql://FfbDMZl571mskftA:MyfJQwpcq82lyJfh@mysql-broker.local.pcfdev.io:3306/cf_05a179d8_9b8f_470f_9b81_14e4b4877c8b?reconnect=true",

 "username": "FfbDMZl571mskftA"

}

ssh into the service instance by providing the server details and the app name that it is bound to.

$ cf ssh -L 63306:mysql-broker.local.pcfdev.io:3306 userservice

vcap@6fe377f0-43cd-4631-641f-c87ac239069e:~$ ls

app  logs  staging_info.yml  tmp

vcap@6fe377f0-43cd-4631-641f-c87ac239069e:~$ 

Happy SSHing into your apps and services.

Using PCF Autoscaler

What is the PCF autoscaler? 

A PCF service which scales to multiple instances based on app usage in the environment. 

How would you use it? 

Attributes that determine autoscaling: 

• CPU usage. 
• Throughput
• Latency

CPU Utilization

• Average CPU percentage for all instances of the app. 

Throughput:

• Total http requests per second (divided by total app instances). 
• No. Of bytes of data that you can move in the network over a given amount of time. 
• A fatter pipe or no. of lanes will help you move more people. 
• Speed test: through put. 
• Download: 50 MB
• Upload: 20 MB

Latency:

• Average latency of applications response to http requests. 
• Does not include router processing or network latency. 
• How long the smallest packet of data it takes to send from point A to point B. 

What can you configure? 

• Set minimum and maximum instance limits.
• Schedule these limits: reduce on weekends, scale up during peak holidays etc., 
• Scaling rules:
• CPU Utilization: 
• Scale up or down depending upon CPU usage in percentage
• HTTP throughput: 
• Scale up or down when the http through put is less than or greater than __ requests per second. 
• HTTP Latency: 
• Scale up or down if latency is less than or greater than __ milliseconds. 

Autoscaling schedules

There is an option to specify min and max limits and schedule autoscale limits based on schedules. This allows you to select the date time and repeatable on specific weekdays.

Note:

• If you manually scale an app, the autoscaler instance unbinds from that app and goes to manual setting. 

Messaging using RabbitMQ and Spring boot

Install RabbitMQ

brew install rabbitmq

PATH=$PATH:/usr/local/sbin

Start the server: rabbitmq-server

Go to home page of rabbit mq

http://localhost:15672/

Provide default userid & password: 

guest/guest

You will be able to login to the Rabbit MQ console to see the traffic and other details. 

Sending a message from the Spring boot application. 

Include the dependency in the Gradle / maven. 

compile('org.springframework.boot:spring-boot-starter-amqp')

ProducerCode: 

/**
 * Created by deiveehannallazhagappan on 3/15/17.
 */
@Service
public class ProducerService {
    private RabbitTemplate rabbitTemplate;

    public ProducerService(RabbitTemplate rabbitTemplate) {
        this.rabbitTemplate = rabbitTemplate;
    }

    public void sendMessage(String messageToBeSent) {
        rabbitTemplate.convertAndSend("deiveeQueue", messageToBeSent);
    }

    @Bean
    public Queue queue() {
        return new Queue("deiveeQueue", false);
    }
}

Create a controller which will trigger sending message to the queue. 

@RestController
public class ProducerController {

    @Autowired
    private ProducerService producerService;

    @RequestMapping("/queue/sendMessage/{messageToBeSent}")
    public String sendMessageToQueue(@PathVariable("messageToBeSent") String messageToBeSent) {

        String timestamp = new SimpleDateFormat("HH:mm:ss").format(new Date());

        producerService.sendMessage(timestamp + ": " + messageToBeSent);
        return "Sent message - " + messageToBeSent + " to the queue";
    }
}

ConsumerCode: 

@Service
public class ConsumerService {
    private RabbitTemplate rabbitTemplate;

    public ConsumerService(RabbitTemplate rabbitTemplate) {
        this.rabbitTemplate = rabbitTemplate;
    }

    @RabbitListener(queues = "deiveeQueue")
    public void processMessage(String message) {
        System.out.println(message);
    }

    @Bean
    public Queue queue() {
        return new Queue("deiveeQueue", false);
    }
}

Send the message by hitting the url

View the message in the console: 

More: 

Changing the default userid and password: 

Reference: https://www.rabbitmq.com/install-homebrew.html

Monolithic vs Microservice

Monolithic application is the traditional application which you normally create (i.e., a single big system) which consists of different components as a single deployable unit.

Example of a monolithic application could be an application that has a front-end, back end system that performs several services.

In this the application is normally deployed as a single unit.

Problems with Monolithic approach:

• A single code base which contains the entire application is difficult for the developers to understand particularly people who are new to the team. The application can be difficult to modify.
• The application takes a lot of time to startup, not to mention the deployment.
• The build of the application takes more time, the larger the code base, the slower will be the IDE, the build sometimes has to build components that are not related to the developer currently working on.
•  Code base maintainence: Maintaining the single code base for all the developer could be a nightmare particularly when a developer committed a wrong code at the end of the day.
• Scalability: You might have only certain parts of the application that need to be scaled, but you might have to scale the entire application.
• Deployment dependency: If you have multiple components and developers in a single code base, you may not be able to deploy or wait for a particular interval to get your code deployed to some environments even though you completed your change faster.
• Every developer need to know the technical stack used in the entire application. Only developers of certain skillset will be able to contribute. Use of a limited technical stack for the entire development team

A lot of problems which are mentioned above can be resolved in the microservice architecture.

A microservice is a relatively small service that can be deployed independently and scaled. This is normally done by a small group of developers who can choose their preferred technical stack to solve a problem.

A big application can comprise of multiple microservices developed by different teams of different skillset. This eliminates any long term commitment to a technology stack.

The code base is relatively small, and hence the build and deployment time is also less. 

Live reload using Spring boot - Dev tools

Spring boot – Dev tools – Live reload

Spring boot dev tools provides some interesting features, this page explains how to use the live reload option.

What is live reload ?

Normally whenever you make any change in the view, you have to do the following changes.

• Make the change in the view.
• Restart / redeploy the application in the server.
• Go to the browser and refresh the page.

Also the above approach works only for the web content, if it is java, you might need to rerun the build, which takes lot of time just to see a simple change.

How to implement live reload in Spring

1.       Include sprint starter for dev tools.

              <dependency>

                     <groupId>org.springframework.boot</groupId>

                     <artifactId>spring-boot-devtools</artifactId>

              </dependency>

2.       Create a simple controller method:

       @RequestMapping("/testlivereload")

       public String  testlivereload() {

              return "Deivee";

      

       }

3.       Install browser extensions for Live reload

https://chrome.google.com/webstore/search/live%20reload?hl=en

4.       Enable live reload

5.       This will be connected to the server

 

6.       Now change the code and save, do not redeploy or start the server.

       @RequestMapping("/testlivereload")

       public String  testlivereload() {

              return "Spring boot - Dev tools - Live reload";

       }

The code will be autodeployed and the browser will be refreshed with the content.

If you are doing frequent changes and you dont wnat the reload to happen every now and then you change the files, you may also have a trigger file which will enable to reload only when the trigger file changes.

You can also set properties to ingnore certain folders in the project, these folders whenver modified will not trigger the reload.

You can also set the following property to disable the livereload
spring.devtools.livereload.enabled = false