Autonomous machinery concept collage
Baidu robotics project: construction, mining, and industrial machinery autonomy.
Robotics Application

Autonomous Machinery System

Applying robotics, AI, and edge computing to make traditional heavy machinery more intelligent, safer, and more productive.

Jason Ye · Baidu project, started 2018 · Engineering interview discussion

1 / 5
Project Context
Background & role

From lab research to real industrial autonomy deployment.

About me

  • Zhixian Ye, or Jason
  • UCSD ECE robotics track
  • Baidu robotics, Sensing World AI/product, GM perception

Why it mattered

  • Labor shortage in harsh industrial sites
  • Improve safety without replacing people
  • Jiangsu waste-factory excavator ran 24/7

Engineering goal

  • Parse high-level customer tasks into common autonomy commands
  • Share perception, planning, control, and safety modules
  • Reuse simulation and replay across machine types
The goal was not to replace people, but to help smaller teams operate heavy machinery more safely, consistently, and profitably.
2 / 5
System Architecture
Robot stack

A hierarchical stack turned operator intent into coordinated base and arm execution.

Human-Robot Interface

Operator dispatches high-level work: move, dig, trench, supervise.

Task Resolver

Parses commands into segments, work cycles, and executable robot behaviors.

Base Loop

  • Encoders, cameras, LiDAR, RTK localization
  • Global route, local mapping, path following
  • Obstacle avoidance while repositioning the machine

Arm & Digging Loop

  • Inclination sensors and hydraulic valve readings
  • Camera/LiDAR material-height and pile-shape sensing
  • Dig, move, dig cycle with calibrated MPC/PID-style controls
3 / 5
Modularization
Technical depth

Modularity made the system testable, portable, and safer to evolve.

1

Hardware boundary

Normalize sensors, drive-by-wire signals, hydraulics, and machine health behind explicit adapters.

2

Autonomy contracts

Keep perception, planning, control, and safety services loosely coupled through versioned interfaces.

3

Safety envelope

Use state machines and monitors so autonomy can degrade gracefully instead of failing silently.

4

Sim/replay loop

Turn field failures into deterministic tests before changing robot behavior in production.

4 / 5
Project Impact
Research to product

The outcome was both technical credibility and business leverage.

Publications
Research visibility

The system generated publishable robotics work and reusable technical narratives.

Patents
Defensible IP

Architecture, safety, perception, or control innovations became protectable assets.

Revenue
Commercial pull

Autonomy moved from prototype value to customer-facing machinery capability.

Platform
Reusable foundation

The modular stack reduced reinvention across machines, sites, and follow-on products.

What I would like to discuss: where the hardest engineering tradeoffs appeared between autonomy quality, safety assurance, and deployment speed.
5 / 5
Saved locally