PHPMeetup - ElasticStack

April 27, 2017

Elastic Stack and the PHPMeetup

So last night I attended my first PHPMeetup, I joined @BurlingtonPHP and spent 3 hours with a great group of people learning about the Elastic Stack from the guys that actually built it.

It was pretty amazing, and it's hard to find the right words to describe it, the crowd, the speaker, the topic and to top it off Pizza and drinks amazing.

The talk was about the Elastic Stack and Jason Greenberg did an awesome job explaining and answering all the questions, he walked us through everything, now and then throwing out Pro Tips which will for sure save a lot of headaches down the road.

Elasticsearch is built for speed and versatility, meaning there are a ton of ways to use it, the basics is that it's a fast searchable data store. When you push some data to it, which is done in JSON format it will store and index it making it readily available to be searched and horizontally scaled across any number of nodes as per the storage requirements.

And from talking with Robert MacLellan also from elastic I learned of all these companies which are actively using their services to power their infrastructure, like Disney, Walmart, Remax and more, it's a powerful and robust system that works well and does what we need it to do, gives us access to data fast and easy in a variety of ways.

Now you ask but how can I use it or learn more about it ?

There are a lot of resources but I would suggest starting with the documentation on their website, but also installing the stack locally. During the meetup I was able to download and start Kibana and Elasticsearch on my laptop while Jay was demonstrating some queries on Kibana (I got a T-Shirt as a prize for that - Thanks Jay).

This is what I did to get it up and running on macOS 10.12.4:

Next unzip all three into a folder on your computer, fire up 3 Terminal windows and navigate to where you unzipped all three of them.

To run Elasticsearch cd into the 'elasticsearch-*' folder and run (the star is dependent on the version you download, 5.3.1 at the time I wrote this)

$ bin/elasticsearch

To run Kibana cd into the 'kibana-*' folder and run (the star is dependent on the version you download, 5.3.1 at the time I wrote this)

$ bin/kibana

And finally I used FileBeat to put some sample data into Elasticsearch so I could play around with it, so this is what you need to do:

Edit the 'filebeat.yml' configuration file to match your system, these are the two important parts:

# Paths that should be crawled and fetched. Glob based paths.
paths:
- /var/log/*.log
#- c:\programdata\elasticsearch\logs\*

And a bit further down the file:

# Array of hosts to connect to.
hosts: ["localhost:9200"]

To run FileBeat cd into the 'filebeat-*' folder and run (the star is dependent on the version you download, 5.3.1 at the time I wrote this)

$ sudo ./filebeat -e -c filebeat.yml

You can always directly talk to Elasticsearch with the URL:

http://localhost:9200/

And then all you have to do is access Kibana at the following URL:

http://localhost:5601

To start using it, make sure you define an Index Pattern using the string below as the 'Index name or pattern':

filebeat-*

And you should be good to go with sample data from your own logs.

I will write another blog post with my experience using Elasticsearch with Laravel.

P.S. - Thanks to Shirley (my love) for giving me the opportunity to attend this and staying home with the kids.

Posted: Apr 27, 2017