Getting Started with Big Data Fundamentals

All of us must have heard these two words nowadays, i.e. Big Data. What is Big Data? Is it a database? Is it storage? There are many people who may not understand the true meaning of Big Data OR who might be assuming it in wrong context. This blog post (and upcoming posts on same topic) is an effort to explain Big Data in simple words. So, let’s begin.

Data Set

What is Big Data?

Does Big Data means data which is big in size? Well, that’s how the name sounds. The term “Big Data” doesn’t actually refer to the size of the data sets, but to the solutions used to extract volumes from the data sets – solutions involving new architectures and technologies.

It does not matter whether the size of data is big or small, these new methods are applicable on every data set. Even if you have small data set, you can use these solutions to manage the data in a better way and extract useful information out of that.

Now the next question is – when should we start using these solutions to get the best out of them?

Well, before I explain the actual factors to be considered, I must say that you should NOT consider “size” as the primary factor for opting big data solutions. The three considerations of Big Data, which is also called concept of three Vs, are:

  1. Volume
  2. Variety, and
  3. Velocity

Volume describes the size, that is, the amount of data generated.

Variety refers to the actual contents of the data set. There could be multiple sources for the data sets and all these sources might be using different formats. That brings variety in the data sets.

Velocity is the frequency at which data is generated, captured and made available for users or other systems for consumption. This is the key factor nowadays which play an important role for opting Big Data solutions. It also lead to evolution of new frameworks, like Apache Spark, Amazon Kinesis, etc.

Continue reading


Using AWS Lambda Function to Create AMI at Runtime

AWS Lambda is a very powerful service. You can write numerous Lambda functions to cater your requirement. This articles explains a real world scenario which uses Lambda function to achieve the intended result.

The Infrastructure in place:

Before I dig deep into Lambda function, let’s understand the existing AWS infra and the requirements. I had a very simple environment with AutoScaling groups configured to take care of Web Servers running on Windows environment. Our SQL Server was running on RDS which we migrated to EC2 recently ( a different story altogether ). Our code was hosted on BitBucket and automatic deployments were configured using AWS CodeDeploy.

The Requirements: 

Our basic requirement was to create a new AMI every time a successful deployment is done. We were also looking to update our AutoScaling group with the new AMI. 

The Solution: 

Here begins the fun part. Let’s start with BitBucket and move with the flow. Once the code is ready to be deployed, go to BitBucket and click Deploy To AWS button. More on configuring BitBucket with AWS CodeDeploy can be found here.

That will pass the control to AWS CodeDeploy which will make the deployment on all the instances currently running as part of Auto Scaling group. Now is the time when you have to think about create an AMI and update Auto Scaling group. To achieve this, I have used Amazon SNS service. After every successful deployment, CodeDeploy service will send a message to SNS which in turn trigger AWS Lambda function and this function will take care of all the requirements we had.

Below is the detailed explanation about this function. Continue reading


Make More Money using Banner Slider WordPress Theme

Hey Folks!!!

I’m here to discuss about making more money using banner ads slider. Believe me, that actually works!

You work hard on your online properties, build a great website or a blog, put in lot of efforts to generate traffic but what NEXT? Continue reading