Real Time Hadoop Architecture

Real Time Hadoop Architecture.

Technology list in hadoop eco system.

Cluster in real time  

Utilizing the beneath you can make it one work stream for getting the logs to appear as measurements

HDFS

MAP Reduce 

Hive 

Hbase 

Solr

Storm

Flume

Kafka

ZooKeeper

Redis 

Solr -- indexing 

kafka -- distributing 

Storm -- processing like map reduce programs for real time events 

Flume cluster -- getting the logs jvm and application logs in real time

Zookeeper -- control all the configurations related jobs 

Hbase -- making as data stored in column oriented way

Hive -- for querying your metrics and display in UI

along with normally people club the CDH(cloudera) model and HDP( Hortonworks)

still in progress....