Please don't let AI "improve" your family photos

Over time, the messy backgrounds in photos become the most interesting parts

Credit: Orion Pictures

Thanks for checking out Nuclear Meltdown. If you want to survive the robot uprising, subscribe now.

When I was growing up in the 1990s, scrapbooking became all the rage. Moms in my suburban town would get together to share tips and tools, like scallop-edged scissors and hole punches that made cute little shapes. I wish I had one of these books handy so I could share what it looked like, but it was a very specific aesthetic — kind of like a pre-internet Pinterest awash in brightly colored construction paper.

My mom went all-in on scrapbooking. She got the specialty scissors and hole punches and stickers. Over time, she made a scrap book for each of her kids. And because she died when I was 20, and didn’t have time for a lot of other creative endeavors as a mom of eight, these scrapbooks are in a way an important part of her legacy. Flipping through their pages, you can still read my mom’s handwritten notes about birthday celebrations and family vacations. It’s clear that the scrapbooks were a labor of love for my mom.

And yet.

In the years since, whenever I’ve dug one of these scrapbooks out of a box in the basement, I am filled not with appreciation but rather frustration and regret. That’s because a large number of the photos in these scrapbooks have been physically cut down, often with cutesy scissors, so all of the context and background is gone. So for example in my scrapbook, you can see me wearing my homemade Halloween costumes, but not where I was or who I was with when I was photographed in those costumes.

Generally speaking, you can’t see the architecture of the houses in the background. You can’t see the clothing of the other people in the scenes. There are no pictures where you can look into the background and see the price of gas on a sign, or marvel at the vintage cars passing in the street. You can’t see the expressions on grandparents’ faces as they watched their descendants play in the yard. You can’t see tables set with Dixie cups and milk that came in paper cartons.

You can’t, in other words, see much of anything at all.

The scrapbooks are definitely an interesting artifact of a 90s trend. But the photos inside are now pretty boring. As historical documents, they’re basically useless.

I was thinking about this recently as I read a piece in Slate from Rebecca Onion about the potential of new and highly sophisticated artificial intelligence tools to edit photos. The peice argues that photos matter for helping people understand the broader stories of their families, which I wholly agree with.

As a parent, I have read advice about the good effects that looking at family photos can have on kids. […] The advice goes like this: Keep family photos around so that your child knows they are a part of a bigger story, and they will develop a sense of belonging, become more psychologically stable, and generally be happier.

But Onion goes on to argue that letting AI airbrush photos into oblivion is a mistake because over time all the messy background stuff offers important context. In fact, that background context often becomes the most interesting part of photos over the long term, and it helps elicit the most poignant memories to boot.

The author uses old family photos to illustrate this point: At a beach, the crowds offer important socio-economic context, revealing that the author spent time at ritzy Martha’s Vineyard, but didn’t have access to a private Kennedy-esque enclave. It’s a fascinating insight, even for someone like me who doesn’t know the author and doesn’t care what she looked like as a child. Another photo shows people on an amusement park ride, with the author pointing out that part of the fun is “screaming really loud with strangers.” If you edit out the strangers, you fundamentally change the experience — and diminish your ability to remember how it actually happened.

In an alternative reality, scrapbooking or AI might have stripped all these details out, rendering the photos more “perfect,” but far less interesting to future observers.

A photo of a day at the beach is one thing, but over the last several days one of the most common uses for AI photo editing technology that I’ve seen is to improve wedding photos. I’ve seen numerous examples of people removing guests, clutter and other elements from pictures of their special day.

Here’s one example in which a photographer almost instantly removed a grandma from a couple’s wedding photo.

@danisorkphotoReplying to @Gina Batista can confirm- there’s nothing it cant do 😧 #photoshopbeta #adobegenerativefill #photoshopai #aiediting

Tiktok failed to load.

Enable 3rd party cookies or use another browser

Here’s another example of that same photographer removing a curb from behind a bride. And here are two more examples of people adding elements to bridal photos. Finally, here’s an example showing how easy it is to remove strangers in the background of a vacation shot.

I can relate to the desire to have better wedding photos. The photographer who worked my wedding was middling at best and we’ve always been pretty disappointed with the visual documentation of that day. Here’s a characteristic shot:

This photo is fine, but what really jumps out to me is the cooler on the bottom right. I think today a lot of people might try to photoshop it out of the frame. And with new AI tools, that’d be easier to do than ever before.

But all these years later, that cooler is actually the most interesting part of the photo. It says something about the era in which it was taken. More significantly, the cooler was there to hold water because we got married amid a brutal and record-breaking heatwave. In the hours after this photo was taken, my wife got heat stroke and became violently ill. It was definitely not ideal at the time, but now all these years later the heat wave and illness have become an important part of our family lore. Our wedding was filled with mishaps — the caterer fell through last minute, the air conditioning at our hotel was broken, etc. — and those mishaps make for a much more interesting story than simply saying “it was a perfect day.”

The cooler is a visual nod to those mishaps, helping my wife and I remember and relate the colorful story of our very imperfect wedding day to our kids. And even after we’re gone and the details of the story are lost, the picture will be a little more interesting as a historical document because there’s a little more of a mess in the background.

All of which is to say, context and background in photos is at least as important as the central subject. And if that context is removed, the photograph loses some of its richest story telling potential.

This doesn’t mean I’m opposed to AI in general¹, and many of the experiments photographers are sharing on social media are objectively cool. And obviously people have been editing their photography since the days of the daguerreotype².

But new AI tools take editing to another level, and when it comes specifically to family photos I’m concerned that over editing may jeopardize our images’ ability to tell stories, preserve history and link the present to the past. And unlike Onion, who offered unedited family photos as positive examples of historical documentation, I can offer the opposite: Scrapbooking as a case study in the damage that occurs when photos lose their context.

Scrapbookers like my mom were well-intentioned. They worked hard on their creations and I celebrate the artifacts that they left us. But with the benefit of hindsight, it’s clear the photos in those scrapbooks would be so much better if they simply weren’t cut up into little pieces. They’d be more interesting, and they’d better help people see where their own story fits in the tale of the larger family group. That might not have been obvious back in the 1990s. But as artificial intelligence now makes it easier than ever before to modify our visual record of the past, I hope we don’t make scrapbooking’s context-cutting mistakes again.

Thanks for reading Nuclear Meltdown. If you enjoyed this post, consider sharing it with a friend.

1. I don’t have Adobe’s new tools, but I have used image generators such as Midjourney, DALL-E and Stable Diffusion, among others. In fact, I use them somewhat regularly to create header images for the articles I write on the housing industry. Here’s a recent example. I see a lot of useful applications for AI imaging tools in media, commercial photography, etc. But I do think something is lost when photos of everyday life end up looking so highly polished that they could be mistaken for commercials or fashion magazine content.

2. All the way back in the Civil War, photographers infamously rearranged bodies on battlefields to create more photogenic scenes. Today that kind of thing is strictly verboten in journalism.