The best books I read in 2016
Read More
In Spark SQL, there are many API's that allow us to aggregate data but not all of the built it methods are adequate for our needs. Fortunately, in these cases we can define our own aggregation functions called User-defined aggregate functions.
Read More
n a recent project, I needed to do some time-based imputation across a large set of data. I tried to implement my own solution with moderate success before scouring the internet for a solution. After an hour or so, I came across this article about the Spark-TS package.
Read More
For the past 6 months or so I’ve been committed to driving the speed limit. Anytime I’ve seen a sign with the speed limit posted I’ve driven that speed or lower. It’s been an enlightening experience for me and I wanted to share my motivation, my experience and some data from that time period.
Read More