Data Engineer, Web Scraping

Washington D.c · United States - San Franciscomid

OtherData EngineeringData EngineerDataData & AI

0 views0 saves0 applied

Requirements Summary

Degree (or equivalent work experience) in Computer Science, Engineering, Information Science,

Technical Tools

OtherData EngineeringData EngineerDataData & AI

Design, implement, and optimize end-to-end data pipelines for scraping and processing structured and unstructured data using Google Cloud Platform (or similar) and best practices;
Conduct ad hoc web scraping and data collection to support research and intelligence initiatives;
Prepare data for further analysis, including data cleaning, transformation, anonymization, and masking;
Contribute to the development of internal and external APIs, following best practices;
Collaborate with ML engineers, other data engineers, and software developers to deliver actionable insights and functional tools, including internal and external dashboards, APIs, and data dumps; and
Drive other critical initiatives.

~1 min read

Degree (or equivalent work experience) in Computer Science, Engineering, Information Science, Data Science or a related field (graduate degree preferred)
2+ years of professional experience in data engineering or a closely related field
Ability to communicate complex technical ideas clearly to non-technical audiences
Proficiency in Python, SQL
Experience with web scraping/crawling (e.g., Beautiful Soup, Selenium, Scrapy)
Experience with Google Cloud Platform (or similar), including storage and database services (e.g., Cloud Storage, CloudSQL, Cloud Spanner) and workflow orchestration (e.g., Cloud Composer/Airflow, Cloud Run, Pub/Sub)
Experience building and managing data pipelines, especially for text data
Comfort working in fast-moving, high-impact environments, such as startups, AI research labs, or security-focused teams