什么是大数据？

大数据 Big Data

据说，词源出自Alvin Toffler，上世纪70年代的作品《第三次浪潮》。

逝者 | 阿尔文·托夫勒：如何化解未来的冲击

虽然大数据是一个泛泛的概念词，但是关于大数据，关于大数据处理分析的话题近来持续升温，现在基本成了新一轮工业革命级别的话题。

大数据是什么，作为数据采集团队，我们很长的时间里一直也在思考，什么是大数据，大数据的前景和价值在哪里。

这篇文章里，我会跟大家一起分享我的看法以及各种有趣的内容和资源，它们关于:

什么是大数据
大数据的实践
大数据的应用场景

最近都在说裁员，如果想知道互联网裁员潮对就业薪资是不是真的产生了持久的负面影响，可以用我们的工具，帮你定时每天采集几次生成列表看一看。

（一）什么是大数据

先听听行家的说法：

大数据就是多，就是多。原来的设备存不下、算不动。

————啪菠萝·毕加索

大数据，不是随机样本，而是所有数据；不是精确性，而是混杂性；不是因果关系，而是相关关系。

_______Schnberger

移步ted：Kenneth Cukier: Big data is better data

America’s favorite pie is
Audience: Apple. Kenneth
Cukier: Apple. Of course it is. How do we know it Because of data. You look at supermarket sales. You look at supermarket sales of 30-centimeter pies that are frozen, and apple wins, no contest. The majority of the sales are apple. But then supermarkets started selling smaller, 11-centimeter pies, and suddenly, apple fell to fourth or fifth place. Why What happened Okay, think about it. When you buy a 30-centimeter pie, the whole family has to agree, and apple is everyone’s second favorite. (Laughter) But when you buy an individual 11-centimeter pie, you can buy the one that you want. You can get your first choice. You have more data. You can see something that you couldn’t see when you only had smaller amounts of it.

曾经人们以为最爱吃的派都是苹果派，不过当你有了更细致的数据，你会发现，苹果派受欢迎其实是一种妥协的结果：苹果派是每个人第二喜欢的口味。

拿到小尺寸派的数据以后你更发现，其实苹果派只能排到第四,第五位的样子了。

你有了更多数据，你就能看到之前你看不到的信息。

大数据最核心的价值是什么？ – 商业 – 知乎推荐@Han Hsiao这篇内容的结构十分清晰，对大数据的正面意义提出了非常清晰地探讨。

大数据听着很牛，实际上也很牛吗？ – 人工智能 – 知乎这里 @陈萌萌说的也特别好，怀疑她是不是真的是一个ai。

大数据最核心的价值是什么？ – 商业 – 知乎，依然是这个问题， @刘飞的文章。

大数据是大数据的采集

大数据行业，本身是依托于数据源存在的服务性行业。

大数据最根本之处在于信息收集方式出现了重大变化与革新。大数据的出现与大量信息直接在网络呈现关系非常紧密。

微博、天猫、淘宝、微信等等都直接产生了大量包括定位、消息记录、消费记录、评价、阅读等等殊为庞大的信息，可以说互联网企业都自然的带有数据企业的标签。不过如果我们从数据的源头看的更仔细一些，还是会发现，其实很多数据依然是有巨大的采集与归类的需求。

Joel Selanikio:Transcript of “The big-data revolution in healthcare”

There’s a concept that people talk about nowadays called “big data.” And what they’re talking about is all of the information that we’re generating through our interaction with and over the Internet, everything from Facebook and Twitter to music downloads, movies, streaming, all this kind of stuff, the live streaming of TED. And the folks who work with big data, for them, they talk about that their biggest problem is we have so much information. The biggest problem is: how do we organize all that information

现在人人都说大数据，但其实大家说的是 facebook，twitter，streaming 等等站点上每天产生的信息，做大数据的人呢，会觉得我们有的数据量实在太大了。

（组织信息仍然是最难的问题）

I can tell you that, working in global health, that is not our biggest problem. Because for us, even though the light is better on the Internet, the data that would help us solve the problems we’re trying to solve is not actually present on the Internet. So we don’t know, for example, how many people right now are being affected by disasters or by conflict situations. We don’t know for, really, basically, any of the clinicsin the developing world, which ones have medicines and which ones don’t. We have no idea of what the supply chain is for those clinics. We don’t know — and this is really amazing to me — we don’t know how many children were born — or how many children there are — in Bolivia or Botswana or Bhutan. We don’t know how many kids died last week in any of those countries. We don’t know the needs of the elderly, the mentally ill. For all of these different critically important problems or critically important areas that we want to solve problems in, we basically know nothing at all.

许多有效的数据还完全不在网络上，要依靠原始的方法来收集。数据方面还有很多基本层面的问题在非常多的领域非常明显。

有哪些「神奇」的数据获取方式？ – Liu Cao 的回答 – 知乎看到这里推荐一个 @Liu Cao

的回答。

严澜(lanceyan)的博客 – 技术分享框架交流大数据处理架构搭建机器人

（二）大数据的实践

工具看这里：大数据分析一般用什么工具分析？ – JavaScript – 知乎

最近看到个例子，说pokemon go 带给玩家运动量上的变化：

1、应用中的数据分析示例：·

六个月以后，大部分pokemon go 的玩家的运动量逐渐和 non-player基本一致了。

看来确实是一个能用相当效果的游戏。

2、交通状况大数据分析示例：

Susan Etlinger: What do we do with all this big data

Now, there’s a group of data scientists out of the University of Illinois-Chicago, and they’re called the Health Media Collaboratory, and they’ve been working with the Centers for Disease Control to better understand how people talk about quitting smoking, how they talk about electronic cigarettes, and what they can do collectively to help them quit. The interesting thing is, if you want to understand how people talk about smoking, first you have to understand what they mean when they say “smoking.” And on Twitter, there are four main categories: number one, smoking cigarettes; number two, smoking marijuana;number three, smoking ribs; and number four, smoking hot women.

这里非常有趣

（三）大数据的应用场景

先贴两个新闻观察：

京津冀大数据产业发展现状 | 报告 | 数据观 | 中国大数据产业观察_大数据门户数据观 | 中国大数据产业观察_大数据门户

如今，在政策上，国家战略层面上，大数据受到的重视程度都越来越高。

应用场景上，现在分布在：