Blog

Augmented Curiosities: An XR Exhibition with the Herskovits Library of African Studies

Augmented Curiosities: Virtual Play in African Pasts and Futures is an immersive exhibition that places African material culture in the hands of its visitors. As an Innovator-in-Residence for Media Technology & Innovation, I developed the project through collaborations with the Northwestern University Library, most specifically the Melville J. Herskovits Library of African Studies, and Northwestern IT. Augmented Curiosities is designed to promote the Herskovits’ incredible object collection and provoke conversations about the use of Virtual Reality (VR) and Augmented Reality (AR) within the storytelling strategies of museums.

I’m an archaeologist and a curator. Within these disciplines, I use material-based methods to explore my interests in expressive and tangible culture as a force of preservation – especially within cultures that do not rely upon the written record. Archaeologists understand that objects and artifacts often embody characteristics of the people and processes that created them. Craftspeople, artists and producers of many practices embed their unique perspectives within the fabric of their works. The energy of an individual, community or landscape can often be gleaned from the materials it produces. Because of this, we curators often use objects to tell stories of the human past and widen an audience’s capacity for imagination. I’m particularly interested in widening imaginations about African pasts and futures. And curation provides me intellectual space to produce scholarship as an artist within an archaeological practice. I work to engage that artistry through experimentation with technology and digital media. Augmented Curiosities is my most recent effort towards that end.

For this project, I sought to find ways to place fragile, valuable or otherwise inaccessible objects from the Herskovits Library of African Studies collection into the hands of visitors. To accomplish this, our team integrated Virtual Reality and Augmented Reality experiences within an exhibition, that allowed visitors to intimately engage the objects. We used photogrammetry and 3D modeling to create a digital replica of each object and the exhibition cabinet in which they are housed. We then made those replicas accessible through the development of a VR application in Unity Game Engine and a web AR experience through the 8thwall platform. This curatorial framework allowed me to express historical, archaeological and anthropological data as time, space and experience. The VR and AR content delivery strategies offer visitors an immersive experience in which they may virtually remove the objects out of the glass-paned exhibition cabinet and play with them. Through this we created a product that was both educational and entertaining – simultaneously exhibiting emerging technologies and relatively hidden objects.

Northwestern University Dean of Libraries, Xuemao Wang, trying out the Augmented Reality Experience

Augmented Curiosities was partly inspired by my interest in affective curation. I believe objects possess energy that humans transform into meaning. We deploy our personalities, emotions and unique lived experiences to make sense of our physical surroundings. I want to curate this phenomenon. Rather than highlighting geographic origins or manufacture dates, I am far more interested in allowing a visitor to have experiences with material culture. I sought to exhibit the feeling of the objects as opposed to prescribing their meanings. To archive this within the exhibition, we produced video interviews from Northwestern University community members that contextualize each object within emotional, political, spiritual and familial landscapes. Interviewees were prompted to respond to the question: “What does this object mean to you and how does it make you feel?” The videos of their testimonies create an atmosphere of intimacy as visitors explore within the VR experience. Affective curation allows me to invert some of the power dynamics of exhibition development, delegating parts of the knowledge-production process to the communities that the exhibition engages. I am eager to further develop this theory and methodology throughout my career.

This experience affirmed for me the expansive opportunities offered by digital curation. The Augmented Curiosities project facilitated the production of a new archive – consisting of 3D models, video, audio and literature. The project created and consolidated a dynamic range of new and existing resources that support cursory browsing or rabbit-hole deep dives into the objects, communities and research that constitute the exhibition. Following the guidance of Esmeralda Kale, the curator of the Herskovits Library of African Studies, I created an Augmented Curiosities LibGuide which preserves the exhibition within the Northwestern University Library database. It is my hope that this rootedness within the digital infrastructure of the Library will encourage further inclusion of digital resources, such as 3D models and expert testimonies, within the University’s digital repositories.

Augmented Curiosities is the product of ambitious ideation and incredible institutional support. An expanse of Northwestern University units contributed time, expertise and funding to produce this unprecedented exhibition. I am grateful and indebted to the talented group of developers, engineers, scholars and technicians that believed in my idea and assisted its creation. Augmented Curiosities is a testament to the worthiness of interdisciplinary research and the ease in which meaningful cross-campus collaboration can occur at Northwestern University.

Personally, this project has been a tremendous success. I tested hypotheses, gained methodological training, developed theory and shared a product of the process with the Northwestern University community – an exemplary graduate student research experience that I’m extremely grateful for. Ultimately, it is my hope that Augmented Curiosities makes evident the power of immersive visualization techniques and encourages visitors to critically reflect upon the value proposition of exhibitions and the museum industry.

Unearthing History from the Skies: Northwestern’s Drone Lab Experience

written by: Craig Stevens, Eli Kuto and Nate Bartlett

In our mission to promote the use of emerging technologies for the Northwestern University community, Media and Technology Innovation (MTI) partnered with the Anthropology Department to bring innovation to the forefront of the classroom. Anthropology Doctoral Students Craig Stevens and Eli Kuto, alongside Nate Bartlett and the MTI team, joined forces to create an unforgettable lab experience for students in the Fall 2023 term Archaeology: Unearthing History (214-0-1) course.

Powered by the whirr of drone propellers and the magic of 3D photogrammetry, we embarked on a mission to unearth the history of our very own campus landscape. In the hands of intrepid anthropology students, our Unmanned Aerial Systems (UAS) weren’t just delivering pizza, they were investigating the history of Northwestern’s iconic Lakefill and capturing the intricate tapestry of the landscape below.

Lab participants waving for aerial drone photograph
Dr. Rozensweig (Anthropology), Nate Bartlett (MTI), Eli Kuto (Anthropology), Craig Stevens (MTI), and ANTH 214 lab section participants waving for an aerial drone photograph.

With the leadership of course instructor Dr. Melissa Rozensweig, the MTI team set out to explore how to integrate the technology in ways that were safe and accessible for students while also figuring out the logisitics of doing this kind of activity with a large-enrollment class. Through consultation and collaboration with the NU Environmental Health and Safety Department (EHS), an extra layer of security was added to our high-flying endeavors.

Our team sought to create a hands-on exploration that highlighted the transformative power of technology in understanding landscapes. To accomplish this, participating students operated a drone in supervised missions, flying gridded patterns over the Lakefill landscape. As preparation, each student read and acknowledged an MTI-crafted safety acknowledgment form – provoking a touch of excitement and responsibility, ensuring students were mentally prepared for their airborne missions.

Over four days, 77 Northwestern students launched a drone from the North lawn of the Norris University Center, investigating and documenting the Northwestern University Lakefill from hundreds of feet above the ground. Their mission was to unravel the historic formation of the campus landscape, using cutting-edge technology to explore the past from an immersive bird’s-eye view.

Guided by key questions on visibility, proximity, accessibility, location, context, and significance, students analyzed a 3D model of the Lakefill, which was made possible by our week-long aerial data collection. The aerial perspective prompted students to contemplate changes, both subtle and profound, in the Northwestern Lakefill and Lagoon. Students compared pre-Lakefill maps with present-day 3D models, witnessing vanished shorelines and the emergence of an innovative architectural marvel. They traced the veins of paths and sidewalks, wondering about the students of the past that once traversed them. This wasn’t just about studying history – it was about feeling its pulse. More than a demonstration of technology, we collaboratively designed a lesson in human-environment relationships and the impact of anthropogenic factors on landscapes. From above, the familiar campus transformed into a living, breathing timeline. And the sculpted earth revealed the fingerprints of human intervention.

screenshot of lakefill 3D model in droneedeploy software
A screenshot of the Lakefill and Lagoon 3D model in the DroneDeploy web interface.

The impact of this lab experience went beyond the classroom. Students didn’t just analyze the Lakefill – they reported on it, creating detailed accounts of its formation and evolution. For homework, students produced a landscape formation report based on their analysis of the 3D models, providing a unique and insightful perspective on the Northwestern Lakefill. Check out some of their enlightening takeaways:

Student A: “From the aerial and 3D view of the Lakefill and lagoon, a lot of the Lakefill area and a significant chunk of campus can be seen through different angles. The aerial view is significant as it can help you judge distance and acreage, elevation levels, plant heights, etc”.

Student B: “The 3D model of the Lakefill emphasizes the man-made lagoon, as the model does not include the entirety of the Lakefill. You gain a clear picture of the density of the foliage, the landscape design of the land near the lagoon, and information on the buildings closest to the lagoon. The aerial view emphasizes the shape of the Lagoon, and its sheer size, marking it as a dominant part of the Lakefill”

Student C: “The aerial view of the landscape provides a detailed overhead view. Using an aerial view displays visibility that is not possible to see from field walking or ground survey. The 3D model centers around the Lakefill and provides imaging of the surrounding landscape”.

Our team was delighted by the first-hand and original analyses of our campus that the students contributed. The lab experience became a pedagogical innovation for Northwestern, showcasing the potential of technology to make education entertaining and impactful.

Lab participants looking up spotting a drone after launch.
Lab participants spotting the drone after the instructional launch.

The collaboration with the Anthropology department is set to continue, with plans in the works to regularly offer this lab activity for the Archaeology: Unearthing History course. Moreover, there is a commitment to finding ways to integrate digital technologies into other course topics, expanding the horizons of archaeological survey and activity areas analysis.

Through this, the ripples of our experiment extended far beyond the Lakefill. Looking ahead, the future holds exciting prospects. Students will have the opportunity to deepen their analytical skills, explore cutting-edge equipment, and delve into the dynamic relationship between technology, data collection, and analysis. As we reflect on the lessons learned, the Media and Technology Innovation team gained valuable insights into student engagement and reaffirmed the potential of drone survey strategies in education and historic preservation.

This rewarding collaboration has allowed the Media and Technology Innovation team to demonstrate the usefulness of emerging technologies for the benefit of Northwestern students, staff, and faculty. As we soar into the future of virtual and immersive experience, we hope this lab experience will be a beacon of inspiration for other departments across the campus to embrace technology and provide dynamic, entertaining and educational experiences for our incredible learning communities.

MTI Virtual Reality Exhibition # 1

This Spring, as the Innovator-In-Residence for Media & Technology Innovation (MTI), I curated our first virtual reality (VR) exhibition in the ENGAGE VR application. In pursuit of our goal of promoting the use of immersive digital storytelling technologies, NUIT and Media & Technology Innovation will be developing rotating virtual exhibitions to engage Northwestern University students, faculty and staff. We will be collaborating with members of the NU community to digitize some of their favorite objects and curate them within dynamic and interactive VR environments. To accomplish this, the project made use of digitization techniques including 3d photogrammetry and modeling, videography and digital curation to produce the inaugural MTI virtual exhibition in the ENGAGE VR metaverse.

Three members of the Northwestern University community, Dr. Amal Hassan Fadlalla, Zoran Ilić and Dr. Mary Weismantel, graciously lended their objects for digitization. Dr. Fadlalla, a University of Michigan Professor, contributed a Hadendowa Knife which she collected in Sudan during the late 1990’s. Fadlalla received her PhD in Anthropology at Northwestern University and donated the knife to the former Chair of the department, Tim Earle. I first encountered the object during an exploration of the Chair’s Office in 1810 Hinman. In addition to its stunning craftsmanship, the knife also encompassed a beautiful narrative of gratitude and exchange. The Anthropology Department stored the knife with a letter penned by Fadlalla in March 1999, which tells the story of the knife’s creation as a gift for Tim Earle and his assistance in helping Fadlalla obtain residency status to pursue graduate studies in the United States. The digital nature of the exhibition allows for audience members to approach the letter, read Fadlalla’s words and intimately experience the materiality of the 3d model.

The exhibition also includes a model of a 1952 edition Croatian Pinocchio Book. This object was donated by Zoran Ilić, Senior Academic Systems Engineer for NUIT and a vital contributor to the production of the exhibition. Ilić and myself conducted photogrammetry on each object in the Emerging Technologies Lab which allowed for the creation of 3d models that can be interacted with in VR. Ilić brought the Pinocchio book to the US from Croatia in the early 2000’s and I was able to interview him on the importance of the object within his family’s cultural heritage. He also took the time to explain the uniqueness of his collection and some of the ways in which the Croatian version of the classic story differs from the popularized Disney-fied edition.

The third object of the exhibition is a Peruvian ceramic Trujillo beer bottle collected by Dr. Mary Weismantel, professor of Anthropology at Northwestern University. This beautifully rich object was contextualized in a short interview with Weismantel, where she explains the symbolism, politics and anti-racism embedded within bottle’s creation. Weismantel is a preeminent scholar of the Andes – learn more about her work and the cultural context of the ceramic bottle in her book Cholas and Pishtacos: stories of race and sex in the Andes.

This project was a pleasure to create. As our first attempt, we were both inspired by the potential of these technologies and also frustrated by some of the difficulties that digitization and photogrammetry present. In the future MTI hopes to improve our processes of data capture and the resolution of our models, through the use of light diffusers, polarizers and tweaks within our studio lab setup. We are also looking forward to hosting the owners of lended objects within the VR application (from wherever they may be in the world) so that they themselves can share their stories and memories with a live VR audience.

I am extremely grateful for the time and patience of the object lenders, and the labor of NUIT staff such as Stephen Poon, Nate Bartlett and Zoran Ilić who assisted with the production of the exhibition. If you are a student, staff or faculty member at Northwestern University and would like to see your one of your favorite objects in a Virtual Reality exhibition, feel free to send an email to craigstevens@u.northwestern.edu

360 VR in the Classroom

  The Emerging Technologies Lab in collaboration with Kellogg School of Management had the opportunity to record a 360 video tour...
Read More

Creative Futures of Generative AI

In September of 2022, nearly two years after its launch, OpenAI made their flagship AI image generation platform Dall-E available to...
Read More

Photogrammetry of a Shiny Object

Photogrammetry is the process of taking multiple images of an object, and processing the images with a computer, in order to...
Read More

Look Your Best Recording Remotely

Prior to the pandemic, our team often recorded on-location or in our studio, using our own cameras and lighting equipment. In...
Read More

2019 Leopold Lecture with Rep. Adam Schiff

Addressing a full audience last night in Cahn Auditorium, California Rep. Adam Schiff, Chairman of the House Intelligence Committee, delivered this...
Read More

Class of 2023 and Transfers Aerial Photo

Welcome Class of 2023 and Transfer Students! Last Friday we were delighted to join about 2000 wildcats to create this year’s...
Read More

Robotics Filming in Tech

Last week, we filmed in the new Robotics and Biosystems Center space in the new AB Fill in Tech, interviewing faculty...
Read More

Media & Technology Innovation at Commencement 2019

Northwestern hosted its 161st annual Commencement this past weekend. Working with the Commencement Office and TC Furlong, Northwestern IT provided a...
Read More

Promotional Videos for Two Northwestern Centers

The video production team has had a very busy spring. It began with our own version of March Madness – creating...
Read More