E.E. Just Goes Digital during COVID-19

The Bridge Builders Foundation’s Ernest Everett Just (E.E. Just) Youth Science Program is designed to expose minority (Black/African American and Latinx) students to careers and academic pursuits in STEM in a hands-on and inquiry-based way. COVID-19, or the novel coronavirus, tried to put a rift in the Summer Marine Science Camp, but the Bridge Builders Foundation did a strategic pivot and engaged a group of partner marine educators to be creative and build a lesson plan and provide instruction to give participating students a hands-on and engaging experience in the digital realm. Using a variety of digital tools and hands-on activities, the educators provided students with a Summer Marine Science Camp experience right in their own homes. One of the hands-on activities even included shipping materials directly to students so they could be the model designers. Pre-Post survey results showed program success in students’ interest and enthusiasm for the program, as well as their conceptual knowledge gain. As part of the data collection and analysis process, we calculated the ‘learning gains’ from the course. A learning gain shows the improvement between the pre-and post-learning assessment scores. Students showed a 41% normalized learning gain, above the 30% minimum significant gain.


Rocky Intertidal Gone Digital!
Just like the intertidal organisms adapt to life in the tide pools, the E.E. Just program adapted to the virtual environment necessitated by COVID-19. Instead of the two all-day in-person workshops during the first week of the program, we shifted to a two-hour class daily over the course of five days to make sure the students didn't suffer from Zoom fatigue. The program was split into two weeks: one week based on the mainland concepts and a second week based on the USC Wrigley Institute Catalina experience. The E.E. Just instructional team was effective in engaging the students using all of the various technology platforms. They transformed a very hands-on and inquiry-based learning environment into an engaging and interactive online learning experience in which the students were responsive in the digital realm. The instructors demonstrated the importance of thinking differently because all of the components of the virtual environment must be scaffolded and properly structured, which is very different from what happens naturally in the in-person classroom space. Students' attention was held by using different techniques and activities and by regularly switching modalities. Various learners were also accommodated by a wide range of activities. One of the common criticisms of online learning is it is often very stagnant and focused on listening; the instructional team consciously picked activities that required student action and creation. Kinesthetically inclined learners were accommodated by dancing and music components like the marine sound activity, which combined movement with snaps, clapping, and sounds. Traditional discussions were moved to chat or made possible by using tools like Padlet and Mentimeter (see details below). Students were also encouraged to share on their own time using Flipgrid video submissions. Creating engaging online STEM lessons requires rethinking some of the standard teaching paradigms we all comfortably use, but it is possible with creative thinking and time.

Learning Goals
Overall the camp works to increase student engagement in science, via marine science, and help them to see science as a possible career that is available to them in the future. Additionally, the camp hopes to increase student connection to the ocean, and help them to see how their individual actions can impact the ocean for better or worse. Specific learning goals for the program are outlined with the key activities in Table 1 below. Students will be able to explain: • There is a marine habitat known as the marine rocky intertidal • Wherever the marine rocky intertidal is found, it has some shared characteristics. • Because it has some shared characteristics, some kinds of animals are found here because they can handle or are adapted to those characteristics. Students will be able to: • Access the learning materials Students will be able to explain: • The rocky intertidal is divided into zones, one of them is known as the mid-intertidal • What an adaptation is • How animal adaptations help them to survive in a habitat Students will be able to: • Access the learning materials Students will be able to explain: • What a tide pool is and where they are found in the rocky intertidal • How tidepools allow some intertidal organisms to thrive outside of the usual zone in which they are found Students will be able to: •

Online Tools and Resources
To create an interactive and engaging learning environment in the digital space, many online tools and resources were used. The class was hosted within Google Classroom, and the sessions during week one were hosted on the Google Meet video conference platform. Students took their pre-and post-tests during the camp on a Google Forms survey. Flipgrid. The Flipgrid platform was used to collect video responses from each of the students and increase student engagement. Flipgrid was used for exit tickets, homework, and even having students demonstrate their knowledge with varying activities. One of the activities on Flipgrid was to create an intertidal song demonstrated by rewriting lyrics to Hamilton and The Fresh Prince of Bel-Air. Flipgrid was a highly effective platform for the team because it included the framework of including not only a prompt, but also an Students will be able to explain: • What a food chain is, what a food web is, how they are different, and how they are similar • Why the intertidal is an important ocean and coastal habitat • How resource availability impacts marine organisms • Some negative impacts on and some behaviors or actions to protect the intertidal Students will be able to: • Access the learning materials Students will be able to explain: • Why the intertidal is an important ocean and coastal habitat • How resource availability impacts marine organisms • Some negative impacts on and some behaviors or actions to protect the intertidal • What a Marine Protected Area is and how they help protect the rocky intertidal and other ocean habitats Students will be able to: • Access the learning materials Students will be able to explain: • Why the intertidal is an important ocean and coastal habitat • How resource availability impacts marine organisms • Some negative impacts on and some behaviors or actions to protect the intertidal • What they have learning over the course of the twoweek program Students will be able to: • Access the learning materials • Successfully participate in the program activities • Clearly and confidently present their PSA on the importance of the Rocky Intertidal • Ask questions of career speaker • Share what they have learned and demonstrate their learning through a variety of formats example video while also providing both grading and peer feedback opportunities. During week one, final presentations were conducted on Flipgrid and played during the synchronous session.

Caption : Intertidal Organism Song Assignment on Flipgrid. Screenshot by Dijanna Figueroa.
Padlet. Another tech resource utilized frequently with this age group was Padlet. Padlet is a multimodal tech platform that allows for a wide variety of student responses. Each of the teachers created their own Padlet to facilitate discussions with students. The teachers created the main static questions at the top of the Padlet and students responded with text, photos, video, or audio recordings. We also utilized it as a digital notebook and we displayed their responses live as they were updated.
Caption : A demonstration of how Padlet was used to facilitate discussions. Screenshot by Dijanna Figueroa.
Mentimeter.com. We used a Mentimeter to demonstrate a live interactive response on both mobile and computer device platforms. We included discussion questions and used students' answers to create word clouds for instantaneous assessment of their understanding of concepts and to build group definitions.

Caption :
A demonstration of how a Mentimeter was used as an assessment tool for prior knowledge.

Hands-On STEM Component
Caption : Worksheet observation and labelling of the Aboral Surface of the Sea Star Model for the dissection. Photo by Tara Treiber.
Sea Star Experiment. The sea star dissection was the most time-intensive component of our teaching, but it also yielded the strongest learning impacts. Students were mailed a paper model of a purple ochre sea star with each of the major organ systems created as a different layer (for example: the reproductive system, the neurological system, the digestive system, etc.). Using a Google Slides presentation to scaffold the dissection, students were introduced to sea stars as a vital member of the intertidal community, and then directed to examine and draw first the oral and then the aboral surfaces. They could choose between drawing their observations free-hand and using a provided worksheet with basic shapes for those who needed that support. Next, students were asked to "cut" the sea star open, by cutting through the tape that was used to hold the paper layers closed, and examine the organ systems layer by layer. The Google Slides showed photos and videos of the actual features on live or actual dissected sea stars that were referenced as they went through the dissection. Key vocabulary was also highlighted and defined in the presentation. Students were then asked to reflect upon and summarize their favorite part of the dissection experience and learning of the day using Flipgrid. This project required the creation of the dissection worksheets and sea star models (which included illustration of the ochre star layer by layer, printing and copying the pieces, cutting them out, assembling the pieces into the sea star), mailing the sea stars and worksheets (stuffing envelopes and addressing and stamping them), developing the the scaffolding Google Slides, leading the dissection, and reviewing the student reflection Flipgrid videos. Tide Pool Maker Project. Not only were the paper sea star dissection materials sent home, but also craft materials were sent for the tide pool maker project. Students were able to make a tide pool environment however they wanted to. They could craft it, build it from natural materials in the backyard, use materials from around their homes, and even draw it. "I liked making the tide pool models with the mollusks, crabs, and sea stars (echinoderms)" said E.E. Just student Liberty Figueroa. The image below includes an example of a drawing as well as thumbnail images of crafted tide pool products the students created.
Caption : A screenshot of Nathaniel W.'s tide pool drawing, which he filmed as a timelapse. Screenshot by Dieuwertje Kast. During the second week, the program shifted gears to best mirror the students' usual Catalina Island Field Experience in the digital realm. The program switched instructors to those affiliated with the USC Wrigley Marine Field Site: Lorraine Sadler, Meghan Macgregor, Natalie Foote, Linda Chilton (provided guidance), Lynn Whitley, Lara Hsia. The second week was also two hours daily over the course of five days. It started with a virtual touch tank tour, where students met living occupants of the touch tank with pre-recorded videos and ended with presentations on Flipgrid from all the students. Each day's content is itemized below.

Day One: A Deeper Dive into the Creatures of the Intertidal
Caption : A screenshot of the touch tank Google Slides activity. Screenshot by Dieuwertje Kast.
One of the highlights of the Catalina trip is visiting the resident touch tank of live oceanic creatures such as crabs, sea stars, urchins, and anemones at USC's campus on Catalina Island. During the field excursion students are able to see these creatures and sometimes hold or feed them in person. This experience was digitized by doing a touch tank tour virtually with professional-quality waterproof cameras that recorded video from inside the tank. The students then observed the creatures in the touch tank and created questions about the organism behaviors. Instructors connected what they were seeing in the videos to the previous week's exploration of the intertidal and extended the scientific learning by organizing the organism exploration by phylum. To summarize and reflect on the learning, students drew a 2D model or created a 3D model of a touch tank creature that they learned about as an exit ticket/homework. This included a typographic drawing as seen below.
Caption : Screenshot of the typographic drawing created by Lorraine Sadler, screenshot by Dieuwertje Kast. To open the day and connect with students' prior knowledge, students were asked to share what they know about plankton on a Padlet. During a regular Catalina field site excursion, students would collect plankton with plankton nets during the day and nighttime and do a comparative microscope analysis of each plankton population. To replicate the plankton experience in the virtual realm, these samples were placed under the microscope and converted to videos that the students watched and worked to assess population differences between various times of the day and locations. Afterwards, students created plankton models (drawn or with craft materials) based on the plankton they saw. Students were given access to a Spark Adobe document about plankton to dive deeper into the topic on their own time.

Day Three: Making Connections -Food Webs and Resources in the Intertidal
The day began with a short instructor-led lesson on the difference between food chains and food webs. After the lesson, students created their own food webs or food chains using rocky intertidal organisms that they have been exploring. Students then explored the food and nutrient resources of the intertidal using an animated video created through a partnership between USC Wrigley and USC Cinema School's Division of Animation and Digital Arts program. Additionally, the video introduced the idea of scientists collaborating with the media and artists to increase their communication reach. To close out the day, guest intertidal scientist Jessie Alstadt talked to the students about her intertidal monitoring work and tied back to their food webs.

Day Four: How Does the Intertidal Connect to ME? Impacts, Benefits, and Sustainability in the Intertidal
Sometimes students wonder why we should care about a habitat or concept presented in scientific learning if its connection to them is not provided. For this day, students explored the importance of the rocky intertidal through a presentation by USC Wrigley educator and researcher Meghan MacGregor on Marine Protected Areas, the benefits of the habitat, and how it connects to students even if they live far from the coast. Students were challenged to create a public service announcement (PSA) using Flipgrid on why they should protect the rocky intertidal.

Day Five: Bringing it All Together
This day was a culmination of the two weeks of learning. Students presented their Intertidal Protection PSAs. One of the students named Merari created a PSA warning the public not to take the animals from the tide pool because they will die. One of the goals of the program is to encourage minority students to see careers in science as a possibility. Because of this, the program has been increasing the number of presentations by people of color in science-related careers. The camp closed out with a presentation by Amber Brown, program administrator for the Tyler Prize for Environmental Achievement. She talked about her experience as a non-scientist who does outreach and raises funds for scientists' work. Students were also introduced to a few black scientists and other possible science role models that they might want to explore in the future. Finally, students completed a Post-Program Survey so we could assess their learning gains over the course of the two weeks.

Data Section
Students who participated in the two-week digital camp had their learning gains documented by their responses to a Pre-Program Survey that was administered on the first day of the program and a Post-Program Survey that was administered on the last day of the program. The Google Forms questionnaire featured both content-related questions and questions about the impact of the program on their lives.
Responses to one of the questions were catalogued with a word cloud as seen below. The answers shifted between the pre-to post-surveys, and we saw a decrease in the "I don't know" or "the ocean doesn't impact my life" answers and an increase in more specific responses based on what they learned.
Caption : Pre-and post-survey word cloud for the "How does the ocean impact your life?" question. Word clouds created by Dieuwertje Kast.

Sea Star Anatomy Identification Infographic:
Students were asked to correctly identify sea star parts on the Pre-and Post-Program Survey, using the same imagery that was used in the Sea Star Dissection Activity. There were huge gains in correct identification of all parts except the Sea Star Arms, as students came to the program with a high level of correct identification skills. Infographic by Tara Treiber. Chart Comparison 2: Student knowledge change for defining an invertebrate. Students chose from: Animal without a backbone or hard skeleton (teal), Animal that lives on the bottom of the ocean (purple), Animal that swims in the open ocean (green), Animal that lives in groups (yellow), and Tiny floating organisms (N/A). Student capacity to successfully define an invertebrate increased from 73.3% of students before the program to 86.7% after the program, a 13.4% increase. Charts created by Tara Treiber.

Chart
Chart Comparison 3: Student knowledge change for defining a phylum. Students chose from: Special place for organisms to live (yellow), A group of organisms with the same body plan (blue), An aggressive group of fish (red), A type of oyster (N/A), and A punk rock band (pink). Student capacity to successfully define a phylum increased from 46.7% of students before the program to 73.3% of students after the program, a 26.6% increase. Charts created by Tara Treiber.
Chart Comparison 4: Student understanding of the importance of the rocky intertidal habitat change over the program. Students chose from: Buffer coastal erosion, Be a safe nursery area for young organismsallowing them to get big enough to survive in the open ocean or other ocean habitats, Be an indicator of climate change, Balance the ocean food web, All of the Above, and None of the Above. Student understanding increased in all categories except the ocean food web. Chart created by Tara Treiber.
The other interesting gains we saw between the Pre-Program and Post-Program Survey were a 20% increase in students saying they liked or really like science and a 23% increase in students saying that they like marine biology. Learning gains can be assessed in a variety of ways, but E.E. Just has been using the normalized gain method as proposed by Richard Hake as a way to evaluate the effectiveness of the program (1998). This year's normalized learning gain showed a 41% increase between the Pre-and Post-Program Survey responses. We also assessed the success of the class using various qualitative methods like attendance and the majority of students stayed throughout the day (attendance wise) for the voluntary program. However, with the shifts in themes and content, it was difficult to do a longitudinal study over the previous years of E.E. Just programs.

Challenges
There were a lot of unprecedented challenges with the transition to the virtual sphere. The first hurdle was regarding student access to technology including devices like Chromebooks, laptops, and ipads and internet connectivity. In combination with the school district, which provided the bulk of the devices that students used, the program delivered devices and hotspots to those who did not have access to either of those. On the first day, the onboarding process took a lot of unexpected time, including entering the digital classroom and getting started, which was comparable to the regular in-person onboarding process. We had to figure out the parent versus child accounts when students logged into the class and keep track of them for attendance. Some of the email addresses did not have real names in the Google Meet classroom, but we fixed this problem during the second week when we transitioned to Zoom. To resolve some of these challenges, we had a designated person who was responsible for connecting parent to child accounts.
Background noise and echoing audio presented another challenge. Siblings who were joining us from the same household on various devices would cause audio reverberation between their devices. Additionally, not all of the students were cognizant of the background noises coming from their homes, which would drown out the instructors who were speaking. Some of the instructors and students would get unstable internet warnings and those resulted in frozen screens and garbled speech. There were issues with battery power and the need for quick access to chargers. We ran into some issues using communal platforms like Google Docs, where some of the students would accidentally delete responses, so we tried to use things like Padlet where that wouldn't happen.

Recommended Changes for Next Virtual Experience
This session of the E.E. Just camp was a major trial run for the instructors trying out various engaging platforms within the digital space.
• Adult to Child Ratios º There was an average ratio of two adults to one student, but there were a lot of adults who didn't have an active role. The basic Google Meet version didn't allow breakout rooms, so we had a lot of adults with no significant role besides serving as observers. We transitioned to Zoom in the second week, which eliminated part of this problem. In the future, we plan to engage the additional adults with a more structured and scaffolded active role. • We recommend: º Being clear about guidelines and expectations of students participating digitally in the class.  Make sure students know that the public chat is not private and to not be discussing nonclass related items on it. º Utilizing digital puzzles and escape room type strategies for motivating assessments and exit tickets. º Increasing the sharing time for students and increasing the time they had to work on their activities and designs. º Allow extra time for prep time ahead of class. With the extreme structuring and scaffolding of the virtual environment, prep time took a lot longer.

Ethics and Consent
The program collects media release forms for all participating students and parental consent for use of students' images and voices, which we utilized digitally throughout the various online platforms.
• Tara Treiber -Treiber was one of the educators who coordinated, developed, and taught the curriculum for the program and helped manage its digital structure; created, distributed, and led the sea star dissection; and is the second author of this journal article.

Author Information
Dr. Dieuwertje "DJ" Kast -Dr. Dieuwertje "DJ" Kast is the director of STEM Education Programs for the University of Southern California's (USC) Joint Educational Project. Through her efforts, she has provided STEM instruction to over 26,000 underrepresented minority students, 600 educators, 25 school principals, and countless community members. She coordinates supplemental science programs in Los Angeles for low income elementary school students of color across a gamut of schools through the Wonderkids, Young Scientists Program, and the Medical STEM Program. She is the co-chair of the Expanding Audiences Committee of the National Marine Educators Association. ORCID ID: 0000-0002-0857-6480 Tara Treiber, MNSE -Tara Treiber is a passionate and professional educator and education consultant, specializing in marine, environmental, and general science education. She believes in the power of people to change the world for the better, with education being at the heart of that change. Treiber has decades of experience turning that vision into action, as a classroom teacher, the education coordinator and member of the start-up staff of the Oklahoma Aquarium, as education director for Heal the Bay and the Heal the Bay's Aquarium, and as an education consultant working for National Geographic Education, Seventh Generation Advisors, Los Angeles Unified School District, and others. Her programs have provided high quality, engaging STEM education to hundreds of thousands of students over the years. She has worked with the Ernest E. Just Youth Science Program for a decade, and she currently teaches high school science and is a teacher leader at New Roads School in Santa Monica, California. ORCID ID: 0000-0001-7225-4518 Dr. Dijanna Figueroa -Dr. Dijanna Figeroa has made a career of exploring the mysteries of the deep. She was featured in James Cameron's documentary Aliens of the Deep, which follows Cameron and NASA scientists as they explore some of the deepest parts of the ocean and learn about the unique life forms that inhabit those spaces. She has participated in hundreds of deep-sea dives using various deep submergence vehicles. Currently, she works at the intersection of science, education, and entertainment. She is a science and nature consultant for various film and media production companies and serves as the director of Education and Playful Learning for the Two Bit Circus Foundation. She's spent more than two decades teaching STEAM to grades K-8 in the greater Los Angeles area, formerly served as global director of the MUSE School National Geographic Society's Green STEAM program, and has advisory roles with many STEM/STEAM nonprofits. She was recently featured on MTV's Women Crush Wednesdays Women in STEM series as a female science "rock star." She teaches middle school science at St. Matthew's Parish School in Pacific Palisades, California and runs programs that teach students how to fly drones, scuba dive, and build underwater robots. If that isn't enough, Dr. Figueroa is the director of the Lucas Scholars STEM Program, a community based social justice and equity program designed to engage young people in science, engineering, design, and art. She has a bachelor's degree in marine biology from UCLA and holds a Ph.D. in marine science from UC Santa Barbara. She is committed to making science, technology, art, engineering, math, and music accessible to all people. She joined the faculty of the E.E. Just Marine Science Camp in 2019. Additionally, as one of a few black women in deep sea marine science, she was honored to have had the opportunity to mentor and share her experiences with young aspiring scientists and explorers of color. She is also a co-chair for the NMEA Expanded Audiences committee. ORCID ID: 0000-0001-7113-1663 Lynn Whitley, M.A. -Lynn Whitley directs pre-college programs for the Wrigley Institute for Environmental Studies at University of Southern California in Los Angeles. She has more than twenty years of experience presenting ocean science and STEM opportunities to students and educators. As co-director for Centers for Ocean Science Education Excellence (COSEE)-West, she worked with scientists and education partners to enhance learning and networking between science educators and research scientists and has developed and