So far, we have covered Blockchain, AI, IoT & Sensors in our weekly educational series. We have had amazing guest blog contributions!
This week, we continue with Big Data. We will give you a (jargon-free) basic overview of Big Data and the key points you should know about it’s potential.
Speaking of Big Data… HM Land Registry are continuously innovating and working on structuring their enormous amounts of data. Be sure to meet their team at Stand 36 at FUTURE:Proptech 2018.
BIG DATA – What is it?
When you are talking about Big Data,
you are really talking about ALL data
Big Data is a term that has been around for some time now but there is still confusion about what it actually is. Big Data works on the principle that the more you know about anything or any situation, the more reliably you can gain new insights and make predictions about what will happen in the future. By comparing more data points, relationships will begin to emerge that were previously hidden, and these relationships will enable us to learn and inform our decisions.
The amount of data available to us is only going to increase, and analytics technology will become more capable. So if Big Data is capable of all of this today – just imagine what it will be capable of tomorrow.
The V’s of Big Data
The History of BIG DATA
For those interested in how we got here, check out this article by FORBES
FORBES, as one of our attending media will also be moderating a panel discussion with Intersection, a Sidewalk Labs company, at 12:20am on the Commercial Stage, titled “Building Cities From The Internet Up!”
The Increasing Volume of Data
- Data is growing at a rapid pace. By 2020 the new information generated per second for every human being will approximate amount to 1.7 megabytes
- In just 5 years the number of smart connected devices in the world will be more than 50 Billion – all of which will create data that can be shared, collected and analyzed.
The below chart from McKinsey shows the value potential of Big Data:
Also from Forbes
Here are some great datasets you can access freely!
BIG DATA – Terms you NEED to know
- Algorithm: A mathematical formula or statistical process used to perform an analysis of data.
- Analytics: Most likely, your credit card company sent you year-end statements with all your transactions for the entire year. What if you dug into it to see what % you spent on food, clothing, entertainment etc? You are doing ‘analytics’. You are drawing insights from your raw data which can help you make decisions regarding spending for the upcoming year.
- Descriptive Analytics: If you just told me that you spent 25% on food, 35% on clothing, 20% on entertainment and the rest on miscellaneous items last year using your credit card, that is descriptive analytics.
- Predictive Analytics: If you analyzed your credit card history for the past 5 years and the split is somewhat consistent, you can safely forecast with high probability that next year will be similar to past years. The fine print here is that this is not about ‘predicting the future’ rather ‘forecasting with probabilities’ of what might happen.
- Prescriptive Analytics: Still using the credit card transactions example, you may want to find out which spending to target (i.e. food, entertainment, clothing etc.) to make a huge impact on your overall spending.
- Batch processing: Batch data processing is an efficient way of processing high volumes of data where a group of transactions is collected over a period of time.
- Dark Data: This refers to all the data that is gathered and processed by enterprises not used for any meaningful purposes and hence it is ‘dark’ and may never be analyzed. It could be social network feeds, call center logs, meeting notes and what have you.
- Data lake: A Data Lake is a large repository of enterprise-wide data in raw format.
- Data mining: Data mining is about finding meaningful patterns and deriving insights in large sets of data using sophisticated pattern recognition techniques.
- Data Scientist: Talk about a career that is HOT! It is someone who can make sense of big data by extracting raw data, massage it, and come up with insights.
- Distributed File System: As big data is too large to store on a single system, Distributed File System is a data storage system meant to store large volumes of data across multiple storage devices and will help decrease the cost and complexity of storing large amounts of data.
- ETL: ETL stands for extract, transform, and load. It refers to the process of ‘extracting’ raw data, ‘transforming’ by cleaning/enriching the data for ‘fit for use’ and ‘loading’ into the appropriate repository for the system’s use.
- IoT: IOT is the interconnection of computing devices in embedded objects (sensors, wearables, cars, fridges etc.) via internet and they enable sending / receiving data. IOT generates huge amounts of data presenting many big data analytics opportunities.
- Machine learning: Machine learning is a method of designing systems that can learn, adjust, and improve based on the data fed to them. Using predictive and statistical algorithms that are fed to these machines, they learn and continually zero in on “correct” behavior and insights and they keep improving as more data flows through the system.
- Stream processing: Stream processing is designed to act on real-time and streaming data with “continuous” queries.
- Structured v Unstructured Data: Structured data is basically anything that can be put into relational databases and organized in such a way that it relates to other data via tables. Unstructured data is everything that can’t ie. email messages, social media posts and recorded human speech etc.
BIG DATA – in just a minute!
Here are a few of the companies that are using Big Data,
check them out whilst at FUTURE:PropTech
Safeguarding land and property ownership worth over £4 Trillion, Their registry contains over 24 million titles, covering 84% of the Land mass.
Datscha collects, matches, aggregates and visualizes data from the best public and private sources.
Realla combines a comprehensive search engine for tenants and investors with powerful agent tools for streamlining & tracking marketing.
Many thanks to our guest blog conributors this week! Check our blog to read latest insights from Vaboo, Cleo Folkes and Infabode – CLICK HERE