ByteDance
Posted 5y ago

训练推理一体化存储研发高级工程师-Data

ByteDance
Beijing, Beijing, China
OnsiteFull Time
Responsibilities
  • designing storage
  • optimizing cache
  • building storage
Requirements
  • 3+ years development experience
  • Strong programming in C/C++/Java
  • Deep distributed systems knowledge
  • Problem-solving and communication skills
Technical tools mentioned
CC++JavaHDFSRocksDBRedisPaxosRaftTCP/IP

Job description

团队介绍:推荐架构团队支撑字节跳动旗下多款APP产品,如抖音、今日头条、番茄小说、西瓜视频、剪映等推荐系统架构的设计和开发,保障系统的稳定和高可用,致力于抽象系统通用组件和服务,建设推荐中台、数据中台;关于在线服务,在这里你有机会参与大规模机器学习在线预估框架的研发与优化,也有机会参与模型训练与调度等相关问题的研究与突破,解决系统瓶颈,降低成本开销;如你对大数据感兴趣,在这里也有机会参与通用实时计算系统的开发、构建统一的推荐特征中台,为推荐业务实现先进的消重、计数、特征服务等;我们期待热爱技术的你加入,一起创造更多可能。

1、负责设计和实现面向推荐/广告大模型推理和训练的多层级存储系统,综合利用显存、本地内存、分布式内存/磁盘、远端大容量存储系统(HDFS)等多种介质进行数据的存储和迁移管理,实现近计算缓存和远端大容量存储的一体化分级系统;
2、负责优化推荐大模型KV Cache命中率,从推理框架、流量调度、多级缓存等多个系统维度入手定制化优化策略,和业务方协作完成端到端推理性能加速;
3、负责建设搜广推通用用户行为数据存储,设计和实现离在线EB级别行为数据的统一存储、IO、近端Cache解决方案,支撑EB级别训练和推理数据,提供高可用保障,在离在线混合负载下,高吞吐和低延时能力的分布式存储系统。

1、3年以上相关领域开发经验,扎实的编程能力,精通C/C++/Java中的一种;
2、对分布式系统的架构和原理有比较深入的了解;
3、优秀的发现和解决问题能力,良好的沟通能力,具备团队合作精神。

加分项:
1、拥抱开源,有开源项目经历,对前沿技术有浓厚的热情和探索欲望;
2、熟悉Paxos/Raft等分布式一致性算法;
3、熟悉分布式事务模型,熟悉常见存储引擎RocksDB、Redis底层代码;
4、熟悉操作系统底层知识,有TCP/IP、IO等系统调优经验。

About ByteDance

Developing AI-driven content platforms and mobile applications.

Similar jobs

Storage Engineer roles near Beijing, Beijing
3w
Save
Mark Applied
Hide
高性能储存研发专家
Shenzhen or Beijing
OnsiteFull Time
X Square Robot
X Square Robot: Develops general-purpose humanoid robots and embodied AI foundation models.
Expert in high-performance distributed storage for AI workloads with systems programming skills (C/C++ or Rust), experience with RDMA/InfiniBand/RoCE, storage kernels, and production deployment.
3FS, JuiceFS, Alluxio, FUSE, VFS, InfiniBand, RoCE, RDMA, GPUDirect Storage (GDS), NVMe, PCIe, DMA, Ceph, Lustre, DAOS, S3, POSIX, Linux Kernel, C++, C, Rust, Python, Go, CUDA
1mo
Save
Mark Applied
Hide
AI存储研发专家
Beijing or Shanghai or Hangzhou
OnsiteFull Time
Xiaohongshu
Xiaohongshu: Social media and e-commerce platform for lifestyle sharing.
3+ YOEBachelor's degree, 3–5 years in AI/data storage or engine development; experience with file systems or high-performance storage preferred; familiarity with Go or Java; strong design, coding, and collaboration skills.
Go, Java, JuiceFS, Alluxio, CPFS, GPFS, Ceph, MinIO
1w
Save
Mark Applied
Hide
存储硬件工程师
Beijing, Beijing, China
OnsiteFull Time
Xiaomi
XiaomiHKEX: 1810: Designs and manufactures smartphones, consumer electronics, and smart devices.
Recent master's graduate in electronics, communications, microelectronics, or a related field with analog/digital circuit fundamentals, semiconductor process knowledge, Cadence proficiency, laboratory skills, and English communication ability.
Cadence, Oscilloscope, Logic Analyzer, PCB
1w
Save
Mark Applied
Hide
存储系统开发实习生
Beijing, Beijing, China
OnsiteInternship
SenseTime
SenseTimeHKEX: 0020: Develops artificial intelligence software and computer vision technology.
Bachelor's degree in computer science or related, C++/Golang experience, knowledge of cloud storage and k8s, strong coding and debugging, documentation skills.
C++, Golang, k8s
1w
Save
Mark Applied
Hide
腾讯云—TurboCFS高性能存储项目
Beijing, Beijing, China
OnsiteInternship
Tencent
TencentHKEX: 0700: Provides integrated internet services, digital entertainment, and cloud technology.
PhD or outstanding MS in computer-related field; strong OS and networking knowledge; familiarity with ext4/xfs and TCP/IP; strong engineering skills in C/C++; experience with Shell, Python or Go.
C, C++, Shell, Python, Go, ext4, xfs, TCP/IP
1mo
Save
Mark Applied
Hide
2027届校招-AI存储固件开发工程师
Beijing or Shanghai or Hangzhou or Xi'an
OnsiteFull Time
Wiscore
Wiscore: Develops compute-in-memory AI chips for edge computing.
2027 graduate with bachelor or above in microelectronics, automation, integrated circuits, or computer; proficient in C/C++; CET-4 English reading; storage research experience preferred.
C, C++
1mo
Save
Mark Applied
Hide
存储系统研发工程师-2027届
Beijing or Shanghai
OnsiteFull Time
MiniMax
MiniMaxHong Kong Stock Exchange: 0100: Global AI foundation model focused on AGI.
Master's or higher in computer-related field (2027 graduates), strong data structures and algorithms, proficiency in C/C++/Rust/Go, understanding of distributed storage and large-model inference.
C, C++, Rust, Go
1mo
Save
Mark Applied
Hide
存储测试工程师(高级资深岗)
Beijing or Chengdu or Shenzhen
OnsiteFull Time
XSKY
XSKY: Provides enterprise-grade software-defined distributed storage and data infrastructure.
5+ YOE5+ years testing distributed file/block/object storage for AI infra; bachelor\u0002s degree in CS-related field; skilled with test design, performance/stability/failure injection, automation, observability and cross-team debugging.
vLLM, TensorRT-LLM, SGLang, Triton, Ray, KServe, RocksDB, Redis, etcd, Ceph, MinIO, S3, MySQL, PostgreSQL, HDFS, Python, Shell, Go, C++, Rust, Linux, Kubernetes, RDMA, NVMe, SPDK, io_uring, NCCL