Fencing Hallucination

Fencing Hallucination offers a human-AI interactive experience that revisits Chronophotography — the first photographic motion visualization technique, invented in the 1880s — in the context of AI image generators challenging the being of a photograph. The system implements real-time Motion Synthesis and Pose-to-Image Translation with neural networks, hallucinating a virtual AI avatar that responds to the audience’s body movement with fencing postures, and poetically visualizing the human-AI interaction as an AI-simulated Chronophotograph. By letting people experience historic Chronophotography resurrected through a subversive AI imaging tool, Fencing Hallucination calls for reflection on the evolving meanings of images.

Chronophotograph of a fencer lunging, the blade tracing a fan of repeated strokes across a black ground — Fencer, Georges Demeny, 1906. Gelatin silver print. Collection of the Metropolitan Museum of Art

Building the pose dataset

Skeletal data can be extracted from fencing video clips using pose estimation models such as OpenPose, to construct a large dataset of fencing poses. That dataset is then used to train the interactive model to generate a responding skeleton, given the user’s skeleton.

OpenPose skeletons overlaid on broadcast footage of a bout at the 2018 World Championships in Wuxi

Two extracted skeletons, green and red, with per-frame confidence scores printed above each fencer

From skeletons to a chronophotograph

Two 3D stick figures facing each other on a dark stage, one mirroring the other's stance — Human audience (left) vs AI (right)

Three-stage diagram: stacks of skeleton frames translated into rendered fencers, then blended into a single chronophotograph — Converting skeletons into images, then into a chronophotograph

System diagram: Kinect captures the audience skeleton, an MLP generates the AI fencer skeleton for Screen 1, while Stable Diffusion translates screenshots into photos blended into the chronophotograph on Screen 2 — System setup. Blue is the realtime interactive loop; orange is the offline pass.

In the room

A visitor mid-lunge between two screens, one showing the paired skeletons and the other the accumulated chronophotograph — The audience fences with the AI; the bout accumulates on the second screen.

Generated chronophotograph: two fencers in white suits repeated across the frame, blades overlapping

Generated chronophotograph with the strokes blurred into a dense white wash where the two figures meet

Combating the “sameness” in AI art

A key research contribution was identifying three structural reasons why AI-generated imagery converges toward uniformity — and demonstrating how interactivity and custom datasets can fight it. This was published as a companion paper on arXiv.

Exhibitions

CHI 2023 — ACM Conference on Human Factors in Computing Systems
NeurIPS 2023 — Conference on Neural Information Processing Systems
Latent Ville — 2023 MAT End of Year Show, UCSB