fieldcady

That's a fantastic quote from him - thanks for bringing it up! I agree with him, except that I wouldn't even go so far as to say the universe seems mathematical; basic physics is the only thing that fits hand-in-glove. Everything else (biology, climate, sociology, etc) becomes either touchy-feely or not mathematically tractable. Math seems to me like an outgrowth of human cognition, although I have no idea what an alternative would be

Sure thing!

1. "Big Data" is right now maybe 50% buzzword, and as such there's no litmus test for it. However, there are two trends the have converged in a big way in recent years, and are collectively called big data. The first trend is the most straightforward; you have more and more data. It becomes "big" around the time that it won't fit on one computer anymore and you start needing to use a cluster to work with it meaningfully, and programming a cluster rather than one computer can be a very different beast. The second trend is that the data is more complicated in its structure. In the past so-called "structure data" was more likely to be a SQL table, with nice orderly rows and columns. "unstructured data" is more likely to include things like a computer log file, documents, or deeply nested data that don't have rows and columns. Several recent pieces of technology, most notably Hadoop, make it WAY easier to process large and unstructured datasets.

2. I'm afraid I don't work much on the pure research end so it's hard to say. But the constant competition between different technologies shows that people haven't really figured out what are the best programming paradigms to use. Map-reduce is less dominant than it used to be, and there's a lot else on the market. Figuring out those best practices is the main hurdle in my mind.

3. Does asking the genie for venture funding count? :) More seriously though, I would probably ask for a way around map-reduce's performance bottlenecks, especially in doing joins

Depending on your background, learn to chug out good code that does non-trivial stuff. That's the biggest thing - I reject interview candidates all the time because they can't do something simple in a real language. Learn to writes code that works, and that is easy to understand/modify. It's amazing how shitty the code is that brilliant people write sometimes, and they end up being useless.

If this isn't a problem for you, then definitely make sure you are familiar with Big Data technologies like Hadoop and Spark.

Finally, I am a huge fan of learning on the job, so try and do internships with real companies if at all possible. I have two masters degrees and I feel like 90% of what I learned before joining the workforce came from two internships at Google.

Ok, "just like any other language" is pushing it a bit, but the connections between math and language are a lot stronger than you might think. The problem is that there is so much stuff in a natural language (culture, physiology of the mouth, etc) that's peripheral to the core syntax, so they look very different on the face. I'm not a linguist, and I don't think I have anything to say that sheds light on that discipline, but I do think that many mathematicians need a reality check.

I don't do any professional work with Excel (I mostly use Google spreadsheets) so I can't say. For SAS and R it depends on how much memory your machine has, and how you're using them. R in particular is easy to start really abusing memory with depending on the library you use. If your data is taking up a significant fraction of your RAM I would start backing away from R.