What is big data: simple about the complex.

October 30, 2020

Have you heard the phrase "Big Brother is watching you"? This is just about big data , or big data. What is it, how is it useful to humanity and how big data works in general - we read and are impressed.

Surveillance is not accidentally chosen as the first association to big data. The point is how this data is collected. And this happens without our consent, imperceptibly and, most likely, we will not be able to fight it in any way. Sounds like a conspiracy theory, right? Don't worry: you won't hide from big data, but if you understand how it works, it will be easier to accept what is happening. Here are some examples of how big data is involved in everyday life:



the bookstore where you just registered or logged in via social network is sending recommendations. Surprisingly, most of the books interest you;

The stocks of detergent you buy in the supermarket near your home are running out, and a discount coupon for the purchase of a month's supply of this powder comes in very handy in the mailing list. The neighbor's coupon does not contain this offer, and indeed there is no powder at all, since she uses the services of dry cleaning;

additional trains appeared in the desired direction. But a few days ago there were no tickets, and there were no new trains on the schedule.

Wonderful coincidences? Divine Providence? No, big data.


How big data works

It seems that it is not too late to define these amazing phenomena.


So, big data is large arrays of unsorted data, as well as a set of tools for processing them. Big data is understood as information about processes, phenomena and actions. This information is accumulated, analyzed (hello, supercomputers!) And allows you to draw quite realistic conclusions. And the more the system “learns”, the more plausible its predictions (hello, machine learning!).


The term "big data" was coined by Nature editor Clifford Lynch in 2008.


How big data works is easy to understand from the example of social rating, which works in China. All actions of citizens are carefully collected: when they wake up and fall asleep, how they get to work, what purchases they make offline and on the Internet, do they have regular partners, what they spend their money on. Any little detail is another touch to a digital portrait of a person. When the minimum amount of data is collected, the most interesting will begin - they will begin to influence the future life of this citizen: will they give a loan or a visa, will they be invited to an important event, how easy it will be to rent a house or rent a bicycle in the park. These decisions will be influenced by big data - collected and carefully processed data.


Big data includes voice recordings, images, video and audio, geolocation, machine (automatically generated) data, search queries, and more.


It turns out that the idea from the TV series "Black Mirror" has become real in a particular country. By 2020, it is planned to fully implement it in all spheres of life in China. It is possible that further this project will spread to other states.


Who collects big data and how

An ordinary person does not have such an opportunity - huge resources are needed. But business and government organizations have these resources. It is a mistake to think that big data does not concern you. If you use the Internet and mobile communications, make purchases in retail chains and present their loyalty cards, even just travel, big data is collected about you at every step.


Big data appears every millisecond. Imagine how many queries its users generate on Google. Plus Instagram, YouTube and Facebook!


Big data is called unstructured data. This means that they cannot be taken and entered into a regular database, where everything is stored in an orderly manner. Imagine an apartment where a little birthday boy has just been celebrated. Leftovers of food, dishes, candy wrappers, wall decorations, gifts, drawings - this is what big data looks like. Complete chaos without a hint of structure. It is either impossible or too expensive to analyze this information with the usual tools. We sweep aside ordinary computers, leaving the computing resources of data centers. It is here that huge amounts of data will be processed and patterns that will help to draw the right conclusions will be found.


Large companies and government organizations can collect, store and analyze big data.


How big data is processed

Let's say we have collected a lot of heterogeneous information and figured out where to store it. What's next? There are technologies that allow you to find the necessary information among the heap of data. This must be done quickly, and the result must be as accurate as possible. The better you do it, the more successful the business that collects this data is. There is still a need for powerful computing resources and software algorithms that are based on the principles of machine learning. People do not work with big data directly - it is long and expensive. The program can catch on one fragment (text, motion, picture, audio, etc. - absolutely any type of data), then on the second, third and so on. She will establish a relationship between them and predict what the next fragments will be.


Examples of big data in business

We have already discussed how we deal with big data in everyday life. It is clear that large companies are behind this. Now grab a few cases, from which it is clear - big data is useful and business needs much more than we do!


Banks use big data to optimize costs and reduce risks. They fight against fraud, assess the solvency of customers, manage personnel, predict the workload of cash registers, branches and terminals.


Manufacturing enterprises use big data to optimize the consumption of materials, form a queue for the purchase of raw materials, predict surges in demand and prices.


Marketing companies predict the success of advertisements for specific users and offer those advertisements that will definitely interest them and help them decide to buy a product or order a service.


Transport companies receive accurate weather forecasts and assess the risks of downtime and delays, monitor the condition of vehicles and start repairs before they fail. Logistic services optimize routes.


The media chooses the best time to publish news - those that will receive the maximum attention and response from readers.


Educational institutions, through the collection of big data, find interesting and useful educational content. And students receive information in the form - text, video, audio - in which they master it better. This increases interest in education and the level of involvement in learning.


The police prevent crime in potentially dangerous areas (by increasing the number of patrolmen) and at events where clashes may occur. The crime rate is decreasing.


Investors use big data to find interesting and promising partners - companies and startups that will bring profit.


Anywhere big data is used to optimize energy costs, logistics, attract new customers, maintain and repair equipment, and reduce losses.


Big data  - is it good or bad?

There is no exact answer. Big data is just a tool, and whether it is good or bad depends on whose hands it falls into. Here are a few examples where big data can be evaluated in different ways.


Good : Search for criminals based on data from CCTV cameras. The system recognizes faces and calculates people who have previously participated in fights, thefts or more serious crimes. And it is also global automation in decision-making - large companies are interested in this. It is easier for them to find an approach to customers, calculate the need for raw materials, predict production volumes and much more.


Bad : intrusive service in any area, lack of basic anonymity. Let's say that before you did not lead a very correct lifestyle, you were interested in prohibited substances, you committed misconduct. Big data has stored this information and drew appropriate conclusions. Now you are not given a long-awaited loan from the bank, you cannot become, for example, a donor. This demotivates, prevents a person from developing.


Neutral : Product advertisements. She often seems intrusive, but! It is better to receive offers that are really interesting than those that you will never use. Agree, it is more correct to show an advertisement for a race car to a businessman on the Forbes list, and not to a school teacher. Another conditionally neutral example is the same social rating. On the one hand, this approach is terrifying and outrageous. But if you look at it from a different angle, it's not so bad. Society becomes more transparent, an imperfect, but at least some kind of, system of justice begins to operate in it.


What will happen to  big data in the future?

Obviously, we will have to accept that the world develops according to its own laws, which one individual person cannot influence. And since big data is at its peak, we should expect that it will even more influence our lives and decision making. It will not be possible to avoid this, but to reconcile, adapt and begin to receive benefits - completely.


The deepest penetration of big data is expected in those areas where it is possible to establish patterns in the external phenomena and actions of people. These are sales, medicine, science, telecom, military affairs, transport, finance. Slightly less active use of big data in education and the public sector. There are more representatives of small and medium-sized businesses who are also interested in the results of working with big data, respectively, the service sector is developing related to the collection of information and the introduction of tools for working with technology. And there is also a growing need for specialists who work with big data. Perhaps such people will very soon appear in your environment!

You Might Also Like

0 comments