1. Motion (major) 🤸

A. Tasks

I focused on generating optimal motions for the below tasks:

• Visible motion to chase moving targets (Ph. D research topic, git (opens in a new tab), paper (opens in a new tab))
• Motion for enhancing color detectability (paper (opens in a new tab))
• Safe travel from A to B
• Distributed motion and task allocation
• Exploration in unknown environments
• Inverse kinematics of manipulators (paper (opens in a new tab))

B. Hardware Targets

The motions were targeted and tested to the below robot types:

C. Backgrounds

• Search / sampling-based planning
• Spline motion primitives such as B-spline, Bezier, piecewise polynomials. (git (opens in a new tab))
• Non-holonomic curves (Dubins, Reeds-Shepp, continuous curvature)
• Optimization-based planning (iLQR, iLQG, DDP, CHOMP)
• Reinforcement Learning (DDPG, PPO)

2. Perception 👀

To implement the motion autonomy in the real-world, I had multiple hands-on experiences in perception algorithms for traversability and localization.

A. Volumetric Mapping

Comfortable with optimization and tuning for the below algorithms to make occupancy from from 3D sensing.

• Octomap and distance field for 3D environments (git (opens in a new tab))
• Voxblox (TSDF, EDSF)
• 3D mesh generation and opengl render from pointcloud (git (opens in a new tab))

B. SLAM

• VIO (vins-mono, ZEDfu) (git (opens in a new tab))
• Graph-based SLAM (RTAB-Map) or Lidar SLAM (LOAM)
• Intrinsic or extrinsic calibration (Kalibr)

3. Reasoning 🧠

In addition to perception (occupancy, ego-localization), reasoning about targets of interest (opens in a new tab) is a must for the aerial chasing system. For real-world experiments, most of the reasoning methods were tested on the Jetson onboard computer.

A. Detection

Hands-on code integration to detect targets from the vision of flying drones. The below algorithms were performed:

• 2D / 3D bounding box from image streams (opens in a new tab)
• Human skeleton detection (opens in a new tab)

B. Segmentation

Given RGB and depth streams, I have used pixel segmentation in RGB and extracted 3D points from depth information.

C. 3D position tracking and prediction

Beyond detection in a single frame, 3D positions of the targets are tracked and predicted to plan the chasing motion of drones.

• 3D Tracking (opens in a new tab): combining classical approaches (color, kalman filtering), I fine-tuned (opens in a new tab) to stably identify the same object even against occlusion and deformation.
• Prediction: based on the tracking algorithm, performed 3d position prediction (opens in a new tab) reflecting obstacles.

4. Design & Integration

A. Mechanical Design

• Part selection: multiple experiences in making multiple drones myself (opens in a new tab), where I deliberated on the selection of battery, actuator, control & computing board, etc.
• Parts design and building: Solidworks design (opens in a new tab) and simulation. Getting my hands dirty with soldering and other hardware stuffs.

B. Software Integration

• Project: Design pattern, cmake structuring for large projects (example template (opens in a new tab))
• Packaging and deployment: Qt for experiment gui (opens in a new tab), Web frontend (link (opens in a new tab))
• Simulation: testing ROS / cmake project in unreal engine using blueprint (my tutorial video (opens in a new tab))

1. Education & Company

• 2013-2015: Started from architecture and architectural engineering @ Seoul National University (opens in a new tab).
• 2015-2017: BS in mechanical & aerospace engineering @ Seoul National University
• 2017-2022: PhD in Robotics in Lab for Autonomous Robotics Research (opens in a new tab), advisor: H. Jin Kim (opens in a new tab). 5 yrs graduation
• 2022-Current: Staff engineer @ Samsung Research, Robot Intelligence Team.

2. Projects

Graduate School

• Autonomous driving in unstructured environments @ Korea Electronics Technology Institute (KETI)
• Multi-fleet exploration for rescue robots @ Korea Institute of Robotics and Technology Convergence (KIRO)

Personal

• Diverse group generation using genetic algorithm (link (opens in a new tab))
• Polynomial trajectory generation with constraints (link (opens in a new tab))
• Inferring attention of a human toward ambient objects using body skeleton and head pose (youtube (opens in a new tab)).

• I love people and enjoy mingling ❤️ (in general)
• MBTI (opens in a new tab) is ESTJ. Love organizing and planning to solve meaningful problems.
• Respect nerds and geeks obsessed with coding, but I am not that kind, and I do not even want to be like them.
• Focus more on why and what. Sick and tired of purposeless work, studying and research. (e.g., writing a paper for making a paper, studying coding for a higher leet code score)
• I think Ph D. guys can be worse than a cleaning worker or a chef, unless their techs could reach and help others in the world.

1. Training Diffusion Model ⭐

I have deep understanding & industrial experiences in diffusion models. In my team, I am working on inpainting / outpainting on-device model, providing clean removal without unwanted object generation even in tight masks. Also, shadow removal is provided in the single model. Please note that random object generations in object removal is a big problem 🗯️ due to the nature of diffusion :(, and my job is to correct it.

I am comfortable with debugging the diffusion (more like DDPM) based models (SD1.5, SDXL) and low-based models such as SD3 and Flux. Especially, I frequently modify their pipes (opens in a new tab) to manipulate denoising steps.

Adaptation from Large Foundation Models for On-Device Applications

I am mainly concerned with on-device application with strict limitations on ROM, peak memory, NPU evaluations, even in the model sizes. To make adaptable LoRA weights from LVMs, I focus on how to get a good LoRA (A, B) with minimum target layers, and did multiple experiments on ranks.

2. Control of Diffusion Models ✏️

Textual Inversion (Prompt Optimization)

Related tech. report (opens in a new tab). Based on the prompt optimization, I compress abstract user intentions (they often do not have an idea on how to put into textual prompts 😼) into embedding spaces, by curating and training models with a good examples.
Also, I made a research how to apply two different intentions by spatially varying latent regions by modifying CFG. The below is an example to embed two user intentions (fill based on the context and clean removal) without text prompts.

Reinforcement

Feeding the high-quality data to make good models, and optimizing task prompts are not enough for a good service. To align various user preferences, I have experiences apply reward-finetuning and reinforcement for non-differentiable rewards.

Company Projects

On-device Model for Generative Edit (Galaxy S25) (opens in a new tab)

• building large-scale training pipeline and dataset curation (more than 55% of total PR)
• Adapting large vision model (LVM) into the target tasks
• Post-training to optimize the final look-and-feel
• Earned high marks in the assessment

Personal Projects

• Virtual influencer with face swap and dreambooth LoRA with Flux.dev (instalink (opens in a new tab), huggingfacePR (opens in a new tab))
• Motion diffusion models (paper under review (opens in a new tab))