121st ASEE Annual Conference & Exposition
June 15-18, 2014
Paper ID #8679
Evaluation of Student Learning Outcomes Due to Self-Guided Engineering
Analysis of Surroundings
Dr. Devin R. Berg, University of Wisconsin - Stout
Devin Berg is an Assistant Professor and Program Director of Manufacturing Engineering in the Engi¬
neering and Technology Department at the University of Wisconsin - Stout.
©American Society for Engineering Education, 2014
Evaluation of Student Learning Outcomes Due to Self-Guided
Engineering Analysis of Surroundings
Inquiry-based learning was explored in an introductory mechanics course through the assignment
of weekly tasks where students analyzed their surroundings within the context of the course
curriculum. The deliverables for each assignment consisted of a photograph or video and a textual
description. Submissions were collected first using course management software and later on the
micro-blogging platform Twitter. Performance for each student was assessed using both course
examinations and a concept inventory quiz. Based on these results it was found that students in
the treatment group did not perform significantly better than students in the control group.
However, the use of Twitter in the course was found to have benefits in terms of real-time student
interaction and immediate instructor feedback.
Inquiry-based learning is an educational approach that allows the student to take ownership over
the education process by self-identifying a problem and formulating their own solution 1,2 . The
application of this method of teaching was explored in an introductory mechanics course taken by
both engineering and engineering technology students.
Students were tasked with applying the principles of fundamental engineering analysis to objects
found in their normal surroundings over the course of the semester. By asking students to
complete assignments where they had to apply engineering analysis to an everyday object, I
intended for the students to look beyond their textbook and relate the course material to their
surroundings. Further, I hoped that students would begin to view their study of the engineering
curriculum as a continuum rather then as a series of discrete pieces which begin and end with the
assignment and submission of homework sets. Similar work by others has demonstrated success
in getting students to make the connection between the classroom and the “real world” 3 .
A preliminary study was conducted using this concept for a single assignment involving static
equilibrium in the same course 4 . Through this effort it was revealed that, in general, students
enjoyed completing the assignment and the ensuing class discussion was more fruitful than with
other course topics. As a result, the concept was adopted more fully into the course with multiple
assignments throughout the semester. The results of this expanded study are presented here.
The deliverables for these assignments consisted of either a photograph or video of an object or
situation that demonstrated the concepts relevant to the week’s course material accompanied by a
brief description of what was depicted in the photograph/video. Examples of students’ work will
be presented along with discussion of lessons learned and recommendations for the use of this
method in the future. Evaluation of student learning outcomes was conducted through the
issuance of pre- and post-assessments using the Concept Assessment Tool for Statics (CATS) 5 as
well as performance on course examinations. Comparisons will be made between a treatment
group, which was subject to the analysis assignments, and a control group, which did not
complete the analysis assignments.
The application of this method of teaching was explored in an introductory mechanics course
taken by students from both an engineering program and an engineering technology program. As
this course is generally taken early in a student’s undergraduate program, they often experience
difficulty grasping the concepts presented and connecting them with real world experiences. To
help promote a deeper understanding of these concepts, students were tasked with taking a look at
their surroundings while considering material presented in lecture. The specific directions
provided are given in Textbox 1. Students were divided into two groups corresponding to
registration in equivalent sections of the same course. One group served as a control group and
the second group served as the treatment group. Both groups were subject to identical curriculum
and assessment with the addition of the assignments described here given to the treatment
Submit one original post per week (photo/video + text) giving an example of something that
demonstrates the concepts discussed in that week’s classes.
Textbox 1: Assignment instructions as provided to the students.
Collection of student submissions took two forms. During the first part of the semester (first three
submissions), student submissions were collected as uploads to a folder on the course’s learning
management system (LMS). During the latter portion of the semester (final four submissions),
Twitter (http://www.twitter.com) was utilized as a means to both collect and promote discussions
around student submissions. On Twitter, students were asked to include a hashtag (#mech2 93)
with each of their posts (tweets) to provide a means of quickly sorting and organizing relevant
posts. The transition from collecting submissions on the LMS to Twitter was made in order to
foster easier out-of-class discussion and communication surrounding a given student submission.
Similar work has suggested that the use of Twitter helps facilitate student engagement outside of
class and potentially improves course performance 6,7 . While the literature on the use of of Twitter
in the classroom is emerging, recent studies have found the platform functional for promoting
concise expression of ideas, critical reading and writing skills, stronger student-teacher
relationships, self-learning in an informal environment, and accountability among other benefits.
Conversely, using Twitter in the classroom has potential disadvantages such as distracting
content, overly constraining character limitations, and privacy concerns 8 . Each of these items
must be considered when assessing the use of Twitter in the classroom and how the integration of
such a tool into the course curriculum might affect student performance.
For both collection methods, students were asked to produce one original submission on an
approximately per week schedule corresponding with the submission deadlines for their normal
homework assignments. Each original submission was expected to include a photograph or video
and a brief descriptive statement that demonstrated the concepts discussed in that week’s lectures.
After the transition to Twitter posts, students were also asked to submit at least two comments on
the posts of their classmates.
To facilitate archiving of student Twitter posts related to the class, all posts containing the
#mech2 93 hashtag were collected and analyzed using the Twitter Archiving Google Spreadsheet
(TAGS) 9 . This tool allowed for automated collection of all tweets tagged appropriately along
with the corresponding time stamp and performed high level analysis of the connections
(mentions) between tweets.
Students in both the control and treatment groups were given both pre- and post-assessments
using the Concept Assessment Tool for Statics (CATS) 5 . A comparison of these assessments
allows for a quantification of any possible variation between groups in terms of concept retention.
The CATS evaluates knowledge in nine areas:
1. Interactions at roller connections.
2. Forces on systems of bodies.
3. Interactions at pin and slot connections.
4. Newton’s 3rd law.
5. Representation of loads at standard connections.
6. Coulomb’s law of friction.
7. Static equivalence between forces, couples, and combinations.
8. Interactions between friction-less bodies in contact.
9. Static equilibrium.
In addition to the CATS, performance on course examinations for each group was recorded as a
group mean and standard deviation.
On the first three assignments, submitted using the LMS dropbox, a total of 79 student
submissions were collected. Consistent with what was found previously 4 , the submissions for this
task gathered from the students exhibited a wide range in terms of quality and imagination. One
example of a student submission is shown Fig. 1. Here it is shown that annotation (as well as
some creativity) was used (Fig. lb) to highlight key features of the bridge truss shown and aid in
relating the photograph to course material.
Figure 1: Student submitted photographs of a bridge truss both (a) without and (b) with annotation.
Other student submissions, such as shown in Fig. 2, included self-constructed free body diagrams
to help explain the subject of the submitted photograph. The inclusion of this additional
information suggested that the student was thinking critically about the situation depicted in their
photograph and attempted to model the situation as seen in lecture.
As per the assignment instructions, students were also asked to submit a textual description of
what they were submitting each week. Similar to the submitted photograph itself, the textual
descriptions also varied in thoughtfulness and completeness. An example of a student submitted
paragraph is shown in Textbox 2 which accompanied the photograph shown in Fig. 3. In this
example, the student constructed a three-dimensional system in equilibrium using a water bottle
and the cord from a pair of headphones. Submissions of this type demonstrated student initiative
to think critically about the course material and then manufacture a situation to fit within the
context of the lecture content for a given week.
For my three dimensional equilibrium photo, I chose to tie an end of my headphones to a hanging
bottle. The headphones were then secured to a cabinet door that created a tension on the bottle.
By the use of construction lines, we can see that the tension on the bottle exists in the direction of
<x,-y, z>, creating a moment at the point from which the bottle hangs <0, 0, 0>.
Textbox 2: Student submitted description to accompany Fig. 3.
For the final four assignments, the method of collecting student submissions was transitioned to
the micro-blogging platform Twitter. During this period, 363 tweets were collected in the archive
over the course of the four assigned Twitter postings with 239 (66%) of them being replies
(comments). It is estimated that approximately 80% of the total tweets submitted for the class
were captured by the archive. Those that were missed were due to the student omitting the
hashtag from their tweet thus making it invisible to the archiving script. It was observed that the
majority of the tweets that were not collected in the archive were comments rather than original
posts. A plot of the the tweet volume versus time is shown in Fig. 4. The four peaks in tweet
volume correspond approximately with the due dates for each of the four assignments. It can also
be seen that between submission deadlines, the tweet volume, while reduced, did not drop to zero.
Indicating that discussions were occurring on the Twitter platform throughout the week. This has
the added benefit of insuring that the course material is constantly being put in front of the
students thus increasing their exposure time to the curriculum.
An example of a student post submitted via Twitter is shown in Fig. 5. One characteristic of the
average student Twitter submission was that the postings were typically simpler than was
demonstrated for the LMS dropbox submissions. This was likely due to the length limitations for
tweets (140 characters) as well as the ease of mobile posting. However, the ability to post from a
smart phone is seen as a strength of this collection method because it helps the student to easily
capture a moment, post it publicly, and receive immediate feedback from his or her peers. Further,
the use of Twitter as a platform made it easier to post other forms of media such as video as
exemplified in Karcheski 10 , Karcheski 11 , Karcheski 12 , and Hertz 13 . The student posts referenced
here made use of Vine (a short-form video sharing site) and YouTube to demonstrate course
content from the student’s surroundings.
Comparing the submissions collected during the first three assignments using the LMS dropbox
|| Weight of Projector (Lb)
Figure 2: Student submitted photograph of (a) an overhead projector support and (b) a free body
diagram representation of the support.
Figure 3: Example of student submitted photo of a constructed system in equilibrium.
Figure 4: Volume of tweets over time.
Figure 5: Example of student submitted photo posted on Twitter 14 .
with those collected on Twitter it is evident that the dropbox submissions often contained more
thoughtful submissions as shown in Figs. 1 and 2. However, it was the experience of the
instructor that this method of submission made it difficult to have meaningful discussions around
the student submissions due to the elapsed time between students completing the assignment and
the following class session. Comparatively, using Twitter as a platform for student submissions
and any subsequent discussion resulted in submissions that contained less detailed analysis (Fig.
5). However, the Twitter platform permitted the conversation to happen in real-time while the
material was still relevant. An example of this can be seen in Trueblood 15 where an initial student
post drew follow up discussion from both the instructor and other students.
Quantitative data was collected from both the treatment and control groups. The results gathered
from both initial assessment and subsequent performance on course exams is presented in Table
1. Data is presented as average group performance for each group along with standard deviation
for each average score. The results show that students in each group performed at a similar level
for the initial assessment. It is apparent that for each subsequent performance evaluation, the
treatment group performed at a higher level than the control group. However, the variations in
performance between each group were not statistically significant based on the standard
deviation. This indicates a correlation between participation in the assigned tasks and course
performance but with a need for further study to improve confidence.
Table 1: Quantitative performance assessment for both treatment and control groups.
22.5 ± 9.7
79.3 ± 10.3
70.9 ± 12.3
76.9 ± 9.6
67.9 ± 11.2
33.6 ± 11.3
31.3 ± 15.4
Using the Concept Assessment Tool for Statics (CATS), students within the treatment (N=36) and
control (N=24) groups were evaluated both prior to treatment and after treatment. A direct
comparison of pre- and post-CATS scores for each group using the group mean is shown in Fig.
6. It is seen that the net gain for each group on the basis of CATS scores alone demonstrates little
difference. The treatment group had a 49.7% performance gain while the control group had a
45.6% performance gain measured as percentage of pre-CATS score.
Breaking down the pre- and post-CATS scores by student and plotting provides the same result as
shown in Fig. 7. The solid line in each plot represents the plot locations where pre- and
post-CATS score were equivalent for a given student. Data points above the line indicate an
improved score with greater distance from the line correlating to greater improvement of score.
As can be seen, there is little measurable difference between the treatment and control groups.
Further, for both groups there was little variation in gains between lower performing students and
higher performing students as indicated by the relative uniformity of score increases across the
range of pre-CATS scores.
□ Pre-test □ Post-test
Figure 6: Comparison of pre- and post-assessment using the CATS.
By looking at student performance broken down by each of the nine categories evaluated by the
CATS, it was found that the greatest performance gains were made in the categories of
“Representation of loads at standard connections.” and “Interactions between friction-less bodies
in contact.” This was found to be true for both the treatment group, with a 202.6% and a 124.7%
increase respectively, and the control group, with a 157.2% and a 131.7% increase respectively,
measured as percentages of the pre-CATS score. This result is likely due to the prevalence of
these two categories throughout the course curriculum.
The use of an inquiry-based learning task in an introductory mechanics course was found to be
helpful for both the students and the instructor. Feedback gathered from the students suggested
that despite some initial skepticism, they found participation in these assignments to be enjoyable
and a welcome deviation from the normal homework routine. There was no clear preference
between the LMS dropbox style of submission collection and the use of the Twitter platform.
From the instructor’s perspective, the use of Twitter has several advantages including automated
collection, distribution, and statistical analysis.
The use of this style of assignment made it possible to follow along with the classes progress
during the semester and attempt to identify misunderstandings early and address them in lecture
immediately. For example, explaining the meaning of an internal moment as related to the student
submission in Fig. 3. Further, the use of Twitter allowed the instructor to answer questions
outside of class more efficiently or ideally allow other students to answer their classmates’
questions. By sorting through the history of Twitter posts 16 , it was found that students also used
Post-test Score Post-test Score
Figure 7: Pre- and post-CATS results for both (a) the treatment group and (b) the control group.
the course hashtag to share links to online resources while preparing for exams as well as to post
YouTube videos as evidence to support their argument for best car make on the basis of quarter
mile time. As the instructor, I occasionally used the hashtag to share interesting links relevant to
the course curriculum to provide students with further reading.
Despite the lack of any clear evidence indicating improved learning outcomes, the use of this
form of student assignment was a positive experience for both the students and the instructor. On
several occasions students would approach me to tell me about something they saw that they
thought was interesting, even when on vacation as shown in the example in Fig. 8. As previously
mentioned, when comparing the LMS dropbox submissions with the Titter posts, it was found
that the Twitter posts in general contained more simplistic analysis of a given situation. However,
the ensuing discussion surrounding a post was often more meaningful and timely than was
possible with the LMS submissions.
As a result of this experience, it is planned to continue the use of this form of journaling
assignment in future offerings of this course as well as in other similar courses. The submission
method used for these assignments will continue to be the Twitter platform for benefits of
real-time communication and automation of the collection and distribution process. One of the
future goals for this work is to use these assignments to encourage students to engage with the
online engineering community through communication with practicing engineers that use Twitter.
This will be done by sharing an instructor curated list of engineers with active Twitter accounts.
Further, since improved student engagement with the course curriculum is expected 17 but not
effectively measured through exams or with the CATS, future efforts in this area will attempt to
quantify student engagement through participation in these activities such as with a self-efficacy
Figure 8: Student submission collected while traveling.
The author would like to thank the UW-Stout students in the Fall 2013 semester sections of
MECH-290/293 for their photographic and textual contributions.
 L Magnussen, D Ishida, and J Itano. The impact of the use of inquiry-based learning as a teaching methodology
on the development of critical thinking. The Journal of Nursing Education, 39(8):360—364, November 2000.
 Michael Prince. Does active learning work? a review of the research. Journal of Engineering Education, 93(3):
223-231,2004. doi: 10.1002/j.2168-9830.2004.tb00809.x.
 Johanna J. Hopp. Journal writing in the physics classroom. The Physics Teacher, 46(7), 2008. doi:
 Devin R. Berg. Experiences with inquiry-based learning in an introductory mechanics course. In 2013 ASEE
North Midwest Section Conference , pages 318-324, Fargo, ND, October 2013. ASEE.
 Paul Steif. Concept assessment tool for statics (CATS), 2010. URL https : //cihub . org/resources/3.
 R. Junco, G. Heiberger, and E. Loken. The effect of twitter on college student engagement and grades. Journal
of Computer Assisted Learning, 27(2): 119-132, 2011. doi: 10.1111/j.l365-2729.2010.00387.x.
 Ilkyu Ha, Jason J. Jung, and Chonggun Kim. Influence of twitter activity on college classes. In Costin Badica,
Ngoc Thanh Nguyen, and Marius Brezovan, editors, Computational Collective Intelligence. Technologies and
Applications , volume 8083 of Lecture Notes in Computer Science , pages 612-621. Springer Berlin Heidelberg,
January 2013. doi: 10.1007/978-3-642-40495-5_61.
 Amandeep Dhir, Khalid Buragga, and Abeer A. Boreqqah. Tweeters on campus: Twitter a learning tool in
classroom? Journal of Universal Computer Science, 19(5):672-691, 2013. doi: 10.3217/jucs-019-05-0672.
 Martin Hawksey. Twitter archiving google spreadsheet TAGS v5, February 2013. URL
 Tyler Karcheski. #mech293 like the ball toss problem, water is shot from a height with a initial velocity, gravity
then acce... https://vine.co/v/hUjFiZiHZvl, November 2013. URL
 Tyler Karcheski. #mech293 the door assist is similar to a 4-link arm, when the electric motor opens the door.
https://vine.co/v/hQUOvpI7xM5, December 2013. URL
 Tyler Karcheski. The fan has a lower velocity when it’s turned on due to inertia and friction. #mech293
#acceleration https://vine.co/v/hTgr6zpuu3j, November 2013. URL
 Dillon Hertz. Noticed that MineCraft simulates gravity rather well http://youtu.be/eqabldwV9Es #mech293,
November 2013. URL https : / /twitter . com/DillonHertz/status/4 0037 432 8765591552.
 Tyler Webb. Saw this as i was walking back from classes, you could analyze the velocity at the top as the arm is
lowered.#mech293 pic.twitter.com/xGdgldiOxc, December 2013. URL
 Olivia Trueblood. Solving the internal forces in a dome would be tricky! the roman’s figured it out though
#mech293 pic.twitter.com/vTmoEiGzw3, October 2013. URL
 Twitter / search - #mech293. URL https : //twitter . com/search?q=%23mech2 93&f=realtime.
 Laura McCullough. Science news and the science classroom. Journal of College Science Teaching , 36(3):
30-33, January 2006. ISSN 0047-23IX.