Fred Benenson spent 6 years doing data science at Kickstarter. When he left last year, he wrote a fantastic recap of his experience.

His “list of things I’ve discovered over the years” is particularly good. Here are a few of the things that resonated with me:

The more you can work with someone to help refine their question the easier it will be to answer

Conducting a randomized controlled experiment via an A/B test is always better than analyzing historical data

Metrics are crucial to the story a company tells itself; it is essential to honestly and rigorously define them

Good experimental design is difficult; don't allow a great testing framework to let you get lazy with it

Data science (A/B testing, etc.) can help you how to optimize for a particular outcome, but it will never tell you which particular outcome to optimize for

Always seek to record and attain data in its rawest form, whether you're instrumenting something yourself or retrieving it from an API

I highly recommend reading the whole post.