It is not unusual to hear a student is taking an advanced placement computer science (AP CS) course these days, but eyebrows raise when Jackeline Mendez tells people about it, because Mendez is a senior at Boston Arts Academy where, as the name implies, the emphasis is on the arts.
“I had a free block, and I was surprised how [computer science is] more than just systems and machines and the Internet,” explains Mendez, who plans to major in physics in college. “People have a mind set that it’s machines, but it’s really not. It’s what the world is right now. We use computer science for everything.”
Her friend and classmate Michelle Romero is also bullish on AP CS, thanks primarily to their teacher, Nettrice Gaskins, who is also director of the school’s STEAM (science, technology, engineering, art, and mathematics) lab. The lab’s mission is to help teachers and students explore the connections between the arts, science, and math, and to incorporate new technology into their projects.
Romero, who plans to major in theater in college, says she became interested in computer science after seeing Gaskins help with projection and lighting from her laptop for a school play. “I thought `oh wow, theater can be three dimensions,’ and adding projections and sensors made it really cool and made the audience enjoy it more,” Romero recalls. “It sparked my interest in wanting to know how … a sensor connected to her computer could make movement and projections.”
Mendez and Romero recently created a short video (https://www.youtube.com/watch?v=uQp-5DdyKGQ) blending technology, physics, and jazz improvisation. The idea was to interpret Einstein’s Theory of Relativity with musician John Coltrane’s jazz using animation and film to create a music visualization based on that intersection. “Anything can be simplified, so instead of doing the mathematical equation where your brain gets confused, we did animations and used computer science technology,” says Mendez. Using Adobe Flash, “Our video explained how Coltrane used his saxophone to explore the connection of jazz and velocity and acceleration, which are vectors. Then we said any moving objects, including sound waves, can be described by the magnitude of its velocity and acceleration as well as direction.”
Gaskins’ class includes students studying visual arts, theater, and music, and all “also have an interest in computer science and are courageous enough to take the step to enter something they’re not familiar with,” she says. “There are new ways to expand computer science and the arts in ways I can’t imagine, and that’s the type of flexibility we have.”
With an increasing number of software programs available to help people create art, the question becomes: is it culturally acceptable for a neural network or machine learning system to produce original artwork?
Jeff Clune thinks so. Clune, an assistant CS professor and director of the Evolving Artificial Intelligence Lab at the University of Wyoming, says there are many reasons why we might want neural networks to produce art. “The main one is because we consider artistic expression as one of the most uniquely human traits. If we want to produce artificial intelligence that rivals human intelligence, that should include art.”
Another reason for artificial intelligence (AI) to produce art is that “it might fuel us to be more creative.” Being exposed to extremely different influences can be inspiring, and is often essential to producing breakthroughs, he adds. “What better way, save for visiting alien cultures, is there to experience truly different ways of thinking and artistic expression than letting AI produce art? I predict it will unlock vast new cultural and scientific landscapes for humans to explore.”
The art produced by networks has greatly informed our scientific understanding of how deep neural networks see the world, says Clune, and they do so very differently than humans do.
He references the Turing Test, and how computers must not only produce art, but “produce art at a level of quality and originality such that it is indistinguishable from art produced by a human” to pass the test. Clune decided to test that theory by submitting art the lab’s deep neural networks (DNNs) produced to the University of Wyoming’s 40th Annual Juried Student Exhibition, which accepted 35.5% of the submissions.
Clune and his team developed an AI “artist” agent and a “critic” or judge agent. The purpose was for the artist to try and produce art the critic/judge would like, he explains. To do that, the team created an innovation engine algorithm with the artist and judge agents inside. The critic was a DNN trained to recognize 1,000 categories of images, Clune says, “and it’s the AI artist’s job to produce images that the critic agrees count as each type of image,” like a motorcycle.
“We have found that it’s important for artificial intelligence, while it’s learning, to have to constantly switch goals and have many goals,” he explains. That is why they challenged the AI artist to make 1,000 different types of images, instead of just one. The team went through the images the artist agent produced and submitted a handful for the competition they thought appeared to show interesting artistic interpretations of concepts, such as a beacon or a prison cell.
“We have found that it’s important for artificial intelligence, while it’s learning, to have to constantly switch goals and have many goals.”
There were no rules forbidding such entries, he notes, “largely because I’m sure the competition organizers didn’t realize it was possible for AI to produce high-quality art. In fact, if the rules for art competitions begin to be rewritten to limit submissions to humans, we will know that AI has become such a great artist that many humans are afraid to compete against it.”
The outcome? Not only were the images accepted, but they were also among the 21.3% of submissions to receive an award. The work was then displayed at the university’s art museum. Clune adds that the judges were evaluating all submitted art on its own merits to see which was good enough to be accepted. “I imagine the judges never considered the fact that the art might have been made by AI.”
Music to the Ears
Perry Cook, professor emeritus of computer science at Princeton University, believes art has been combined with technology longer than many people realize. “The march of music has been about technology. Music has, in some cases, brought about new technologies, and certain music has taken advantage of the technology of the day,” says Cook, who is also a research coordinator and IP Strategist for SMule, a social network based on creating and sharing music, and co-founder and executive vice president of Kadenze, an online arts and technology education startup. In a blog post (http://bit.ly/2feVCyd), Cook cited numerous artists who have created artwork using machine learning, which he says opens up all kinds of new possibilities.
Although a common perception is that computer music came along late in the evolution of electronic music, according to Cook, people were looking at using computers to store and analyze music in the 1950s. “Forward to today, the computer is just another technology, in my opinion. It’s somewhat comparable to putting a valve on a bugle so you could play more notes, [which] allowed music to get louder and softer.”
The arts are an especially good way to introduce people who normally would not have thought they should or would want to learn concepts about computer science, he notes, specifically artists, children, and underrepresented minority groups “who think they’re kind of locked out or aren’t interested and don’t want to be engineers or computer scientists. Getting to them through the arts is a way to open up [computer science] to a whole new community.”
Making CS “Culturally Responsive”
That is exactly what Gaskins is doing at the academy. She feels strongly that the way to engage her students who come from various ethnic and socioeconomic backgrounds, in science and math is through STEAM. Her approach to her AP CS class is “culturally responsive computer science,” a concept that uses examples from different cultures to create computational artifacts, she says. Gaskins got the idea from a program at Rensselaer Polytechnic Institute (RPI) that shows teachers and students how to see algorithms from a cultural perspective, using Culturally Situated Design Tools (CSDT), and thus, learn principles of engineering, science, math, and computing.
During research Gaskins conducted as a doctoral student, she noticed “there were certain groups that were less represented” in STEAM or STEM, women being key among them. There were also several ethnic groups that were underrepresented: African Americans, Latinos, and native Americans or indigenous groups, she says.
A colleague of Gaskins’ created a software tool to look at local or cultural knowledge and merge it with CS, and saw positive results, “which led me to believe that the content is the problem; the content isn’t engaging the populations that aren’t well represented.” Teaching CS with culturally relevant or responsive content, she believes, makes a huge difference.
Her course is not designed to be a traditional computer course, she emphasizes, and she teaches the principles using 3D modeling, visual arts, music—anything a student might find relevant.
Ron Eglash, a professor in the departments of Science and Technology Studies and Computer Science at RPI, developed CSDTs as a way to teach anyone how to create their own simulations and artifacts. In a TED Talk (“The Fractals at the heart of African designs,” http://bit.ly/2e5l0ne), Eglash explained his focus in the U.S. was on African-American, Native American, and Latino students. “We’ve found statistically significant improvement with children using this software in a mathematics class in comparison with a control group that did not have the software. So it’s really very successful teaching children that they have a heritage that’s about mathematics, that it’s not just about singing and dancing.”
Gastrins’ approach to her AP CS class is “culturally responsive computer science,” using examples from different cultures to create computational artifacts.
For Gaskins, creating art through culturally response computer science is not only acceptable, it is an imperative if the U.S. is to stay competitive in STEM, as well as STEAM.
“If a student has no exposure to computers and they enter a situation where there’s nothing in a course that’s familiar to them, they’re likely to drop the course,” she says. “There’s a lot of people who don’t know they can be interested because they don’t know anything about it, and if you unlock that for them and show them how relevant it is no matter their culture … you’re opening the door for them to come into the subject.”
Further Reading
Nguyen, A., Yosinski, J., and Clune, J.
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images, Computer Vision and Pattern Recognition (CVPR ’15), IEEE, 2015
Ahmed, S.U., Camerano, C., Fortuna, L., Frasca, M., and Jaccheri, L.
Information Technology and Art: Concepts and State of the Practice, Norwegian University of Science and Technology, Norway, http://bit.ly/2eYgBBq
Bond, G.,
Software as Art, Communications of the ACM 48(8), 2005, pp. 118–124
Mitchell, W.J., Inouye, A.S., and Blumenthal, M.S.
Beyond Productivity: Information, Technology, Innovation, and Creativity, The National Academies Press, 2003.
Machin, C.
Digital artworks: bridging the Technology Gap, Proceedings of the 20th Eurographics UK Conference, IEEE Computer Society, 2002.
Figures
Figure. Algorithmic innovation engines are capable of producing images that are not only given high confidence scores by a deep neural network, but are also qualitatively interesting and recognizable.
Figure. Images produced with innovation engines were not only accepted to a selective art competition and displayed at the University of Wyoming Art Museum, but they also were among the 21% of submissions that won an award.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment