ADAS Data Development - backend direction (16394)

主要职责/Your Responsibilities:

Technical key words:

1. Manage and optimize distributed computing tasks, building high-throughput data ingestion, computation, storage, and query services.
2. Develop APIs for replay, data management, and reporting systems.
3. Design and optimize data models using SQL and NoSQL databases, and integrate them with existing legacy Java systems.
4. Using Docker and Kubernetes to deploy and manage services on cloud platforms
5. Data Pipeline Development: Build and optimize multimodal sensor data preprocessing pipelines.
6. Optimize Data Logger software, responsible for efficiently collecting and caching sensor data from peripherals such as cameras, LiDAR, radar, GPS/IMU, etc.
7. Build reliable solutions for online and offline data transmission

岗位要求/Required Qualification:

1. Master’s degree in Computer Science, Electrical Engineering, Robotics, or a related field (or equivalent practical experience).
2. Work Experience: 8+ years of professional backend development experience, with at least 4+ years focused on large-scale data platforms or AI/ML infrastructure.
3. Proven track record of designing, building, and maintaining production-grade data platforms handling petabyte-scale or larger datasets (preferably in autonomous driving, robotics, or geo-spatial domains).
4. Hands-on experience with both backend (microservices, API gateways, async task queues, streaming pipelines) and frontend (e.g., React/Vue, dashboarding) development.
5. Deep expertise in large-scale compute cluster management – e.g., orchestrating Kubernetes, Slurm, or proprietary schedulers over multiple GPUs (NVIDIA A100/H100, AMD MI series), including job prioritization, preemption, auto-scaling, and resource fragmentation optimization.
6. Experience with data lifecycle management – tiered storage (hot/warm/cold), data versioning, lineage tracking, and efficient retrieval for training and replay workflows.
7. Familiarity with ROS/ROS2, ADAS data formats (e.g., MDF, ASAM OpenDRIVE), and annotation tooling integration is a significant plus.
8. Prior involvement in cross-functional collaboration – working closely with perception engineers and fleet operations to translate algorithm needs into scalable platform features.
9. Backend: Expert proficiency in Python and/or Go; experience with frameworks like FastAPI, gRPC, or Django. Solid knowledge of distributed messaging (Kafka/Pulsar) and stream processing (Flink/Spark Streaming).
10. Frontend: ability to build intuitive internal tools
11. Data Storage: Deep experience with columnar formats (Parquet, Iceberg, Hudi) and query engines (Trino/Spark). Knowledge of object storage (S3/GCS) and distributed file systems (Lustre/Weaver) for high-throughput I/O.
12. Compute Orchestration: Proven ability to design and tune job schedulers (Slurm, AWS Batch, or custom K8s operators) for mixed workloads (training, simulation, data preprocessing). Experience with GPU provisioning, GPU resource shcedule platform-like system.
13. Infrastructure-as-Code: Terraform, Ansible, or similar; strong grasp of monitoring/observability (Prometheus, Grafana, OpenTelemetry) and cost management for cloud/hybrid environments.