Awesome-Articulation-Reconstruction-Works

This is a repo that collects some works about articulated object reconstruction. Articulation reconstruction is about reconstructing interactive objects from observation gathered from interaction with the object. I will also write blog posts for some works that contains outstanding approaches. Please post an issue if there are more papers or projects that you recommend.

Keywords: Digital twins, Articulation reconstruction, part-level reconstruction, Interactive Perception

Reconstruction

Ditto: Building digital twins of articulated objects from interaction. CVPR 2022.

[Github] [Project]
This paper propose a pre-trained model to reconstruct articulated physical objects. based on the point cloud of the object before and after interaction based on point segmentation. It can divide object into 1 static part 1 moving part, and a revolute or prismatic joint. However, it is limited in generalization to unseen object categories and it does not address detailed appearance reconstruction.
Blog post TBD.

Ditto in the house: Building articulation models of indoor scenes through interactive perception. ICRA 2023.

[Github] [Project]
This paper proposed an approach to reconstruct articulated objects based on interactive perception. It takes observation as input and performs affordance prediction to choose an interaction point. Then by performing the interaction the articulation inference gets the observation before and after interaction, which are used to form the articulation model along with contact region. However there are no planning module for embodied AI in this paper, agents are teleported to the interaction point.
Blog post TBD.

PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objects ICCV 2023.

[Github] [Project] [Video]
This paper introduces a self-supervised, end-to-end architecture that learns part-level implicit shape and appearance models and optimizes motion parameters jointly without requiring any 3D supervision, motion, or semantic annotation. The training process is similar to original NeRF but and extend the ray marching and volumetric rendering procedure to compose the two fields.
Blog post TBD.

ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory ICRA 2021.

[Github] [Project]
This paper proposed a method to predict articulation model based on a sequence of depth images. The paper uses screw theory to unify the the representation of different articulation types. No segmentation of static and dynamic part is included in this method.

FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects RSS 2022

[Github] [Project Page]
This paper proposes a 3D Articulation Flow to model the articulation property of objects and guides robot manipulation. It proposes a network to predict the flow based on point cloud input and guides the manipulation of robot.

Interactive Perception

Interaction-Driven Active 3D Reconstruction with Object Interiors SIGGRAPH 2023

[Github(Coming soon)] [Project Page]
This paper proposed a method automate the interactive process and segment multiple dynamic parts with multi-round interaction. By encoding the input registered point clouds of the object, it generates the planned motion based on where2act and reconstruct the links. The point clouds before and after interaction are labeled and concatenated to encode as pointwise features by PointNeXT and concatenated with motion parameter, a residual MLP is used to generate the mask of segmentation. After segmentation, the point cloud is completed by SeedFormer and use pre-trained neural dual contouring (NDC) to mesh.
Blog post TBD.

Neural Field Representations of Articulated Objects for Robotic Manipulation Planning