Russell Chen

A/B Testing, Machine Learning, Causal Inference, ETL, Data Engineering, Agentic AI, Generative AI

Python (scikit-learn, numpy, pandas, xgboost, pytorch, shap), R (tidyverse, caret, glmnet), Git (GitHub)

Data Visualisation & Dashboarding: matplotlib, ggplot, Tableau, Shiny, Mode, Streamlit, Looker

Hadoop, Spark (PySpark), SQL (Snowflake / Hive / Presto), Docker, Airflow, Prefect, Jenkins, AWS, GCP, dbt

Block, Inc. / Cash App

• Created a metrics repository that grew to 800+ metrics used in experiment measurement, working with data scientists companywide to provide guidance on metric definitions and data governance.
• Orchestrated all metrics and experiment data pipelines to ensure reliability, making appropriate changes for scalability as overall data processing volume grew to ~10TB per day, while managing compute costs.
• Liaised with third party experiment assignment vendor Amplitude to ensure our requirements are met and to clarify details of assignment mechanism as necessary.
• Collaborated with Eng to build

  Exposium

  , a comprehensive internal dashboard that standardized calculation of experiment results and reporting, combined with robust monitoring and alerting, obviating the need for standalone dashboards. This gave us a bird's-eye view of the experiment program, enabling us to correct deficiencies such as deviations from experiment best practices.
• Used LLMs to automatically summarize experiment results in

  Exposium

  based on raw metric movements.

Robinhood

• Worked with engineers and other data scientists on our internal experimentation platform

  Kaizen

  to:
  • Manage the 1000+ metrics in the metrics repo to ensure that high quality, actionable metrics are used. We also completed a migration to metrics that are defined fully in code and entirely self-serve.
  • Apply ML to estimate heterogeneous treatment effects (HTE) so that experimenters can understand how different segments of users respond to product changes, paving the way for personalized experiences.
• Started and supervised Data Science Experiments oncall to help with experiment design and

  Kaizen

  usage.

NextRoll

• Oversaw ad performance (CPM, CPC, budget fulfillment, etc.) for major external clients in NextRoll Platform Services, including Yelp and Rakuten. I worked closely with engineers to proactively monitor and debug issues relating to segment population, campaign creation, unpausing, optimization and spend pacing leading to positive feedback, improved relationships and renewal of $500K+ platform fees in total.
• Responsible for all reporting for NextRoll Platform Services. Created and maintained the most widely used dashboards for this new team; orchestrating the ETL and defining new aggregated data cubes.

Verizon Media Group / Yahoo! Inc.

• Worked on improving various aspects of our internal experimentation platform

  Evaluate

  :
  • Implemented a bucket size calculator to suggest appropriate control/test bucket size at experiment setup in order to ensure that the A/B test has sufficient statistical power to detect changes in key metrics.
  • Led the development of Ready-to-use A/A buckets. This modifies the process of allocating users to experiment buckets in order to minimize pre-existing differences between buckets, thereby allowing experiments to start immediately. This work is patented and was presented in a talk at IEEE Big Data 2017.
  • Used variance reduction techniques to improve experiment sensitivity / increase statistical power.
• Member of Experimentation Council overseeing A/B tests companywide, from planning to decision review.
• Carried out a thorough Yahoo! Finance user segmentation study, facilitating subsequent work in
  • Daily dashboards giving product managers greater visibility into trends in key metrics for all segments.
  • Lookalike modeling using xgboost to acquire new users in ad campaigns using creatives tailored to the segment being targeted. Resulting incremental new installs / onboarding worth $1M+ in LTV.