Witmem
Posted 1y ago

推理框架工程师

Witmem
Beijing or Shanghai or Hangzhou
OnsiteFull Time
Responsibilities
  • developing framework
  • optimizing performance
  • supporting hardware
Requirements
  • Bachelor in computer-related field
  • 5+ years C++ development
  • Experience with PyTorch/JAX and MLIR/Triton/XLA/IREE
  • Deep learning model compilation/optimization, and heterogeneous compute optimization
Technical tools mentioned
C++PythonPytorchJAXMLIRTritonXLAIREECUDAROCm

Job description

1. 开发训练/推理框架,参与训练/推理框架对于不同硬件的支持和优化;
2. 从工程和算法层面对训练/推理框架进行优化,能解决精度和性能问题;
3. 参与训练/推理相关功能产品化,为客户提供稳定高效的解决方案;
4. 开发推理引擎的运行时系统。包括内存管理以及资源管理等等;

1. 计算机相关学科本科及以上学历;
2. 熟悉C++、Python等编程语言,5年以上C++开发经验;
3. 熟悉深度学习框架 Pytorch、JAX;
4. 有使用 MLIR/Triton/XLA/IREE 进行AI模型编译和优化的经验;
5. 熟悉计算机体系结构基础知识,有扎实异构计算优化(GPGPU/x86/ARM等)、高性能网络架构通信优化、分布式训练策略优化等方面的经验;
6. 熟悉多层次的软件优化经验,如分布式优化(PD分离)、存储优化、kv-cache优化、动态Batching、IO优化、算子/指令集优化等;
7. 熟悉 CUDA/ROCm Runtime和Driver PI及其底层实现,理解常用机制如异步launch,事件event,进程隔离/调度。
8. 有很好的团队协作能力与沟通能力,对技术和代码品质有追求。

About Witmem

Develops computing-in-memory AI chips for edge devices.

Year founded
2017
Employees
300
Organization type
Private
Latest investment
Raised $200.00M Series B (2023) — led by 国投创业
Headquarters
CN

Similar jobs

Inference Framework Engineer roles near Beijing, Beijing
2w
Save
Mark Applied
Hide
推理框架研发工程师-深圳/北京/上海
Shenzhen or Beijing or Shanghai
OnsiteFull Time
Tencent
TencentHKEX: 0700: Provides integrated internet services, digital entertainment, and cloud technology.
4+ YOE3+ years experience; proficiency in C/C++, Python; experience with large-model inference frameworks, distributed inference, GPU/AI chip optimization and parallel strategies.
C, C++, Python, vllm, sglang, tensorrt-llm, NVLINK, GPU RDMA
1mo
Save
Mark Applied
Hide
推理服务框架研发工程师-Data AML
Beijing, Beijing, China
OnsiteFull Time
ByteDance
ByteDance: Developing AI-driven content platforms and mobile applications.
Strong Linux and C/C++ skills, solid data structures and algorithms, multithreading and concurrency expertise, experience with high-concurrency distributed services, good communication and problem-solving; GPU and Redis/RocksDB/gRPC experience a plus.
Linux, C, C++, Redis, RocksDB, bRPC, gRPC, GPU
2mo
Save
Mark Applied
Hide
顶尖应届-大模型推理框架研发工程师-MiMo AI 基建
Beijing, Beijing, China
OnsiteFull Time
Xiaomi
XiaomiHong Kong Stock Exchange: 1810: Designs and manufactures smartphones, consumer electronics, and electric vehicles.
Master's in CS/ECE/math, strong C/C++ and Python, solid data structures and algorithms, CUDA/GPU and kernel optimization, PyTorch/Megatron/vLLM/SGLang experience, distributed systems knowledge (MPI/NCCL).
SGLang, C, C++, Python, CUDA, PyTorch, Megatron, vLLM, MPI, NCCL
1y
Save
Mark Applied
Hide
推理框架开发工程师/专家
Beijing or Shanghai or Hangzhou
OnsiteFull Time
Infinigence
Infinigence: AI infrastructure provider optimizing heterogeneous computing for large models.
Experience building high-performance inference frameworks; proficient in C/C++ and Python; knowledge of large-model inference techniques and distributed optimization; familiarity with vLLM, TensorRT-LLM, or similar preferred.
C, C++, Python, vLLM, SGLang, TensorRT-LLM, Ascend, NCCL, HCCL, PagedAttention, FlashAttention, KV Cache
4y
Save
Mark Applied
Hide
推理框架工程师
Beijing or Shanghai
OnsiteFull Time
MiniMax
MiniMaxHong Kong Stock Exchange: 0100: Developer of multimodal artificial intelligence foundation models and applications.
Bachelor's in computer or related field; expert in performance analysis, computer architecture, C/C++, Python; strong systems reliability and monitoring experience.
C/C++, Python
3mo
Save
Mark Applied
Hide
AI推理框架工程师-2027届
Beijing or Shanghai
OnsiteFull Time
MiniMax
MiniMaxHong Kong Stock Exchange: 0100: Global AI foundation model focused on AGI.
2027 bachelor or higher in computer science or related; strong performance analysis, systems and architecture knowledge; proficient in C/C++ and Python; GPU performance tuning; service reliability experience.
C, C++, Python, GPU
6mo
Save
Mark Applied
Hide
MiMo-大模型推理框架开发工程师
Beijing, Beijing, China
OnsiteFull Time
Xiaomi
XiaomiHong Kong Stock Exchange: 1810: Designs and manufactures smartphones, consumer electronics, and smart hardware.
Master's degree in CS, software engineering, or math; strong programming and deep learning experience; proficiency in Python, C++, CUDA; experience with PyTorch and large-model inference frameworks.
vLLM, SGLang, Python, PyTorch, C++, CUDA
3w
Save
Mark Applied
Hide
【27届校招】大模型训练推理框架工程师
Shenzhen or Beijing
OnsiteFull Time
XPeng
XPengNYSE: XPEV: Designs and manufactures smart electric vehicles and autonomous technology.
Master's degree or above in computer science or software engineering; proficient in Python and C++ or Go; experience in distributed training, GPU/HPC, ML platforms, or inference optimization; understanding of large-model and RL training.
Python, C++, Go, Megatron-LM, DeepSpeed, FSDP, CUDA, NCCL, RDMA, Kubernetes, Ray, Airflow, TensorRT, vLLM