Russell Chen

A/B testing, User Segmentation, Machine Learning, Statistical / Mathematical Modeling

Python (scikit-learn, numpy, pandas, xgboost, tensorflow/keras), R (tidyverse, caret, glmnet)

matplotlib, ggplot, Tableau, Redash, Shiny

Hadoop, Spark (PySpark), Hive / Presto / SQL, Docker, Airflow, Jenkins, AWS EC2 / S3 / EMR

Robinhood

• Worked with engineers and other data scientists on our internal experimentation platform

  Kaizen

  to:
  • Manage the 1000+ metrics in the metrics repo to ensure that high quality, actionable metrics are used. We also recently completed a migration to metrics that are defined in code and entirely self-serve.
  • Prototype methods to estimate heterogeneous treatment effects (HTE) so that experimenters can better understand how different users respond. This paves the way for personalized experiences in the product.
• Started and supervised Data Science Experiments oncall to help with experiment design and

  Kaizen

  usage.

NextRoll

• Oversaw ad performance (CPM, CPC, budget fulfillment, etc.) for major external clients in NextRoll Platform Services, including Yelp and Rakuten. I worked closely with engineers to proactively monitor and debug issues relating to segment population, campaign creation, unpausing, optimization and spend pacing leading to positive feedback, improved relationships and renewal of $100K+ platform fees in total.
• Responsible for all dashboards and reporting for NextRoll Platform Services. Created and maintained the most widely used dashboards for this new team, scheduling parts of the ETL, defining new data cubes.

Verizon Media Group / Yahoo! Inc.

• Worked on improving various aspects of our internal experimentation platform

  Evaluate

  :
  • Implemented a bucket size calculator to automatically suggest to experiment owners how large their control/test buckets should be, depending on the website being tested. This is done at experiment setup in order to ensure that the A/B test has appropriate statistical power to detect changes in key metrics.
  • Led the development of Ready-to-use A/A buckets. This modifies the process of allocating users to experiment buckets in order to minimize pre-existing differences between buckets, thereby allowing experiment owners to start their A/B test immediately, bypassing 4 days of traditional A/A validation and decreasing Type I error rates. This work is patented and was presented in a talk at IEEE Big Data 2017.
  • Used variance reduction techniques such as CUPED (Controlled experiment using Pre-Experiment Data) to improve experiment sensitivity / increase statistical power.
• Member of Experimentation Council overseeing A/B tests companywide, from planning to decision review.
• Carried out a thorough user segmentation study (k-means clustering and PCA) in collaboration with Yahoo! Finance product managers. This foundational research facilitated subsequent work in
  • Daily dashboards giving product managers greater visibility into trends in key metrics for all segments
  • Lookalike modeling using xgboost to acquire new users in ad campaigns using creatives tailored to the segment being targeted. Resulting incremental new installs / onboarding worth $1M+ in LTV.
• Successfully mentored a Statistics Masters student intern who returned full-time after graduation.

Walmart Global eCommerce

• Scored Sam’s Club members’ propensity to buy various items, as input to numerous marketing campaigns.