Skip to main content

Posts

Showing posts from 2020

K8s External Secrets integration between AWS EKS and Secrets Manager(SM) using IAM Role.

What is K8s External Secrets and how it will make your life easier? Before saying about External Secrets we will say about k8s secrets and how it will work. In k8s secrets we will create key value pairs of the secrets and set this as either pod env variables or mount them as volumes to pods. For more details about k8s secrets you can check my blog http://jinojoseph.blogspot.com/2020/08/k8s-secrets-explained.html   So in this case if developers wants to change the ENV variables , then we have to edit the k8s manifest yaml file, then we have to apply the new files to the deployment. This is a tiresome process and also chances of applying to the wrong context is high if you have multiple k8s clusters for dev / stage and Prod deployments. So in-order to make this easy , we can add all the secrets that is needed in the deployment, in the AWS Secret Manager and with the help of External secrets we can fetch and create those secrets in the k8s cluster. So what is K8s external Secret? It is an

LXC Containers and How it differs from VMs and Docker Containers?

VM vs LXC ########## The main difference between the VMs and Containers is that virtual machines require their own kernel instance to run while  containers  share the same kernel. In other words,  containers  are virtualization at operating-system-level whereas  VMs  are virtualization at the hardware level. In virtual Machines each Guest OS will have its own Operating System and kernel. But in the case of LXC Containers the Guest OS will share the same OS of its Host machine. LXC vs Docker ############# LXC is a container technology that gives us the lightweight Linux containers and Docker is single application virtualisation which is based on top of the containerd Run time engine.. Thus even though based on container utilisation they may sound similar but they are completely different when it comes to usage. Unlike the lightweight LXC containers, docker does not tend to behave lightweight VM’s. Single application restriction of the docker is strictly by the design We can easily log o

K8s Secrets Explained.

 What is Secrets in K8s? Secrets are used to store sensitive information like Authentication Token, Passwords, SSH Keys and  Certificates. It is stored in etcd, datastore of k8s. The main advantage of the secret is that it can be updated dynamically , so if we want to change the username or password or ssh key of any of the containers inside the pod, then just changing the secret will dynamically update the existing and newely created POD's env values and ssh-key (mount as volume) . jinojosep@cloudshell:~ (boreal-physics-256910)$ kubectl create secret generic secret-demo --from-literal=username=jinouname --from-literal=password=jinomypassword secret/secret-demo created ################################ jinojosep@cloudshell:~ (boreal-physics-256910)$ kg secret NAME                  TYPE                                  DATA   AGE default-token-l5tmg   kubernetes.io/service-account-token   3      14m secret-demo           Opaque                                2      8s ###############

Updating the Jenkins Version

Common location of jenkins war file on ubuntu server is: cd /usr/share/jenkins/ Stop the jenkins server $ sudo service jenkins stop Move existing jenkins war file $ sudo mv jenkins.war jenkins.war.old Download latest/specific jenkins war file $ sudo wget https://updates.jenkins-ci.org/latest/jenkins.war # For latest $ sudo wget https://updates.jenkins-ci.org/download/war/2.222.4/jenkins.war # for 2.222.4 Start the Jenkins server $ sudo service jenkins start

Grok pattern for custome java log

Grok pattern for the below log. 27-05-2020 06:44:33.476 [app-api-5bd9d99b8-sjql5-6f5bdf4a-f2c9-4a25-8fe6-031e9fa28cf0] DEBUG 1 [http-nio-8080-exec-4] c.w.w.m.customer.controllers.CustomerController [get-141] : Get all Customer request received. This has to be added in the logstash config file /usr/share/logstash/pipeline/logstash.conf filter {       grok {         match => { "message" => ["%{DATE_EU:date} %{TIME:logTime} *\[%{DATA:requestId}] %{LOGLEVEL:logLevel} %{NUMBER:processId} *\[%{DATA:threadName}] %{JAVACLASS:className} *\[%{DATA:origin}] :%{GREEDYDATA:message}"] }       } } alternative grok pattern ##################### (? %{MONTHDAY}-%{MONTHNUM}-%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}.%{NONNEGINT}) *\[%{DATA:requestId}] %{LOGLEVEL:logLevel} %{NUMBER:processId} *\[%{DATA:threadName}] %{JAVACLASS:className} *\[%{DATA:origin}] :%{GREEDYDATA:messagebody} Example logstash file with grok pattern for parsing ############################

ELK Stack with Filebeat using Docker Containers

ubuntu@master:~$ git clone https://github.com/deviantony/docker-elk.git ubuntu@master:~$ cd docker-elk ubuntu@master:~/docker-elk$ sudo docker-compose up -d Creating dockerelk_elasticsearch_1 ... done WARNING: Image for service kibana was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`. Creating dockerelk_elasticsearch_1 ... Creating dockerelk_logstash_1      ... done Creating dockerelk_kibana_1        ... done ubuntu@master:~/docker-elk$ sudo docker-compose ps           Name                         Command               State                                   Ports                                ------------------------------------------------------------------------------------------------------------------------------------------- dockerelk_elasticsearch_1   /usr/local/bin/docker-entr ...   Up      0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp                      dockerelk_kiban