Kevin Weng, Examining Conversational Programming Design Needs with Convo, a Voice-First Conversational Programming System Using Natural Language, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2020.
Abstract: The emergence of voice-based technology and voice assistants has paved the way for new opportunities to democratize learning computational thinking and programming skills, especially for people who do not have access to the traditional programming experience, due to external circumstances or disabilities. In this thesis, I created Convo, a voice-first conversational programming system that allows users to cre- ate programs and develop programming skills simply through natural interactions and conversations with a programming agent and voice feedback. Additionally, I conducted a user study to study the effectiveness of voice-first conversational pro- gramming systems like Convo as well as receive user feedback to explore the design needs of such systems. The user study involved forty-five participants ranging from fourteen to sixty-five years old completing tasks in Convo using and comparing three different input modalities - using just voice inputs; using just text inputs; and using both voice and text inputs. The participants answered questions about their experi- ences with each input modality and general feedback on conversational programming systems. Results showed that participants have preference towards text and found voice-based programming the most difficult to use among the three input modalities. However many participants, especially those new to programming, saw the value and future potential of voice-based programming, especially when speech recognition be- comes more accurate. Additionally, I created design recommendations based on the results for conversational programming systems, including being flexible and accessi- ble, reducing cognitive load and improving speech recognition and natural language understanding.
Jessica Van Brummelen, Kevin Weng, Phoebe Lin, Catherine Yeo, (2020 in submission). CONVO: What does conversational programming need? An exploration of machine learning interface design. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) in submission.
Vast improvements in natural language understanding
and speech recognition have paved the way for conversational
interaction with computers. While conversational agents have
often been used for short goal-oriented dialog, we know little
about agents for developing computer programs. To explore the
utility of natural language for programming, we conducted a
study (n=45) comparing different input methods to a conversational
programming system we developed. Participants completed
novice and advanced tasks using voice-based, text-based, and
voice-or-text-based systems. We found that users appreciated
aspects of each system (e.g., voice-input efficiency, text-input
precision) and that novice users were more optimistic about
programming using voice-input than advanced users. Our results
show that future conversational programming tools should be
tailored to users programming experience and allow users to
choose their preferred input mode. To reduce cognitive load,
future interfaces can incorporate visualizations and possess
custom natural language understanding and speech recognition
models for programming. Preprint
Phoebe Lin, Jessica Van Brummelen, Galit Lukin, Randi Williams, Cynthia Breazeal, (2020). Zhorai: Designing a Conversational Agent for Children to Explore Machine Learning Concepts. Tenth Symposium on Educational Advances in Artificial Intelligence (EAAI).
Understanding how machines learn is critical for children to develop useful mental models for exploring artificial intelligence (AI) and smart devices that they now frequently interact with. Although children are very familiar with having conversations with conversational agents like Siri and Alexa, children often have limited knowledge about AI and machine learning. We leverage their existing familiarity and present Zhorai, a conversational platform and curriculum designed to help young children understand how machines learn. Children ages eight to eleven train an agent through conversation and understand how the knowledge is represented using visualizations. This paper describes how we designed the curriculum and evaluated its effectiveness with 14 children in small groups. We found that the conversational aspect of the platform increased engagement during learning and the novel visualizations helped make machine knowledge understandable. As a result, we make recommendations for future iterations of Zhorai and approaches for teaching AI to children. Peer Review Paper
Jessica Van Brummelen, Catherine Yeo, Kevin Weng, (2020). Learning to program conversationally: A conversational agent to further democratize programming. 14th annual International Technology, Education and Development Conference (INTED2020).
Interacting graphically, as in App Inventor, is no longer the only primary mode of human-computer
interaction. With the rise of conversational agents, like Amazon Alexa and Siri, many people
interact using voice. Its convenience and efficiency make it a viable method for sending messages
and gathering information. For similar reasons, we see voice as an effective method to lower the
barrier to entry to programming and teach an even wider audience computational thinking (CT)
skills. Imagine telling your computer to make your toy race around a track while it plays your
favorite song—and it actually happening!
We developed a voice-based, conversational agent to do just this: empower nearly anyone to
program—just by having a conversation. The system is capable of program creation, debugging
and execution. Peer Reviewed Abstract
Nikhil Bhatia, Using Transfer Learning, Spectrogram Audio Classification, and MIT App Inventor to Facilitate Machine Learning Understanding, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2020.
Abstract: Recent advancements in deep learning have brought machine learning and its many applications to the forefront of our everyday lives. As technology has become more and more integrated into our educational curriculum, researchers have focused on creating deep learning tools that allow students to interact with machine learning in a way that incites curiosity and teaches important concepts. My research contribution focuses on applying transfer learning and spectrogram audio classification methods to teach basic machine learning concepts to students. I introduce the Personal Audio Classifier (PAC), a web application that allows users to train and test custom audio classification models that can classify 1-2 second sound bites recorded by the user. Alongside this in-browser machine learning tool, I provide a set of best practices for spectrogram audio conversion in machine learning applications. I also contribute a custom App Inventor extension that allows users to use the output of the web interface to create App Inventor applications that rely on their trained custom audio classification model. Finally, I provide a high school workshop curriculum based on PAC and the App Inventor extension and detail the results of running workshops with three high school classes at the Boston Latin Academy. My results show that high school students with no prior machine learning knowledge are able to grasp important technical concepts related to machine learning and its applications, as well as build and explore custom machine learning models in the browser through the exploration of a hands-on curriculum based around PAC.
Natalie Lao, Nikhil Bhatia, (2020). Using Transfer Learning, Spectrogram Audio Classification, and MIT App Inventor to Facilitate Machine Learning Understanding, International Conference on Computational Thinking Education 2020 (CTE2020).
Abstract: Recent advancements in deep learning have brought machine learning and its many applications to the forefront of our everyday lives. As technology has become more and more integrated into our educational curriculum, researchers have focused on creating deep learning tools that allow students to interact with machine learning in a way that incites curiosity and teaches important concepts. Our research contribution focuses on applying transfer learning and spectrogram audio classification methods to teach basic machine learning concepts to students. We introduce the Personal Audio Classifier (PAC), a web interface that allows users to train and test custom audio classification models that can classify 1-2 second sound bites recorded by the user. We also contribute a custom App Inventor extension that allows users to use the output of the web interface to create App Inventor applications that rely on their trained custom audio classification model. Peer Reviewed Paper
Natalie Lao, Irene Lee, Hal Abelson, (2020). Experiences from Teaching Actionable Machine Learning at the University Level through a Small Practicum Approach, International Conference on Computational Thinking Education 2020 (CTE2020).
Abstract: Machine learning (ML) courses have traditionally been taught through a math-first approach. They generally begin by establishing mathematical theories behind ML, such as the perceptron algorithm, logistic regression, and backpropagation, and then use these building blocks to motivate more complex structures such as neural networks. Such educational resources may not be sufficient or preferable for audiences who wish to use ML to build useful artifacts but do not have a strong mathematical or programming background. In this paper, we introduce a new framework for teaching actionable ML that combines three components in a Use-Modify-Create progression: (1) technical modules taught through hands-on labs, (2) a capstone project, and (3) supplemental lectures for new areas of research. This framework was applied in two iterations of a semester-long practicum at Massachusetts Institute of Technology (MIT) as a beginner-accessible course aimed at helping a broad range of students gain the ability to ideate and implement independent ML projects. We present the curriculum, student projects, pre and post- course survey responses, assignment grades, reflective discussions, and learnings from both iterations of the course. Our results indicate that the proposed actionable pedagogical framework for ML along with the content and practices of the course were effective for increasing students’ practical self-efficacy in ML and computational identity as developers of ML applications. The findings of this study illuminate patterns of interaction with ML systems that support a practical approach to teaching ML in order to increase knowledge acquisition, future learning ability, and motivation in beginner students. Peer Reviewed Paper
Fred Martin, Irene Lee, Nicholas Lytle, Sue Sentance, Natalie Lao, (2020). Panel: Extending and Evaluating the Use-Modify-Create Progression for Engaging Youth in Computational Thinking, SIGCSE ’20 (March 11–14, 2020, Portland, OR, USA).
Abstract: The Use-Modify-Create progression (UMC) was conceptualized in 2011 after comparing the productive integration of computational thinking across National Science Foundation-funded Innovative Technology Experiences for Students and Teachers (ITEST) pro- grams. Since that time, UMC has been widely promoted as a means to scaffold student learning of computational thinking (CT) while enabling personalization and allowing for creative adaptations of pre-existing computational artifacts. In addition to UMC’s contin- ued application, it has recently been utilized to scaffold student learning in topics as diverse as machine learning, e-textiles, and computer programming. UMC has also been applied to instruc- tional goals other than “supporting students in becoming creators of computational artifacts.” This panel will re-examine the UMC progression and refine our understanding of when its use is suitable, and when not, and share findings on evaluations and extensions to UMC that are productive in new and different contexts. Organized Panel
Jessica Van Brummelen, (2019). Conversational Artificial Intelligence Development Tools for K-12 Education. AAAI Fall Symposium.
Abstract: Artificial intelligence (AI) education is becoming increasingly
important as the adoption of AI technology, including
conversational agents such as Amazon Alexa, Siri, and
the Google Assistant, increases. Current educational and programming
tools enable non-programmers to develop simple
conversational agents, or advanced programmers to develop
complex agents. However, to the author’s knowledge, there
are no tools for non- or novice programmers to develop conversational
agents for the purpose of learning AI and programming
skills. This paper describes AI curriculum that includes
content about conversational agents, machine learning
(ML), and AI ethics, as well as a blocks-based conversational
AI interface developed within MIT App Inventor. During a
series of six workshops, students used this interface to develop
conversational agents for social good, including a memory
aide, math tutor, speech visualizer, and recycling assistant.
In this paper, I present (1) the blocks-based interface,
(2) the conversational AI curriculum, (3) how conversational
AI directly relates to computational thinking skills,
and (4) results from an initial small-scale study. The results
show that through the curriculum and using the blocksbased
conversational AI interface, students learned AI and
ML concepts, programming skills, and to develop conversational
agents for social good. Peer Reviewed Paper
Jessica Van Brummelen, Judy Hanwen Shen, Evan W. Patton, (2019). The Popstar, the Poet, and the Grinch: Relating Artificial Intelligence to the Computational Thinking Framework with Block-based Coding. International Conference on Computational Thinking Education 2019 (CTE2019).
Abstract: As the world becomes increasingly saturated with artificial
intelligence (AI) technology, computational thinking
(CT) frameworks must be updated to incorporate AI concepts.
In this paper, we propose five AI-related computation
concepts, practices, and perspectives: classification;
prediction; generation; training, validating, and testing;
and evaluating. We propose adding them to a widely-used
CT framework. We also present an MIT App Inventor extension
that explores this framework through projectbased
learning. Through the framework and practical
block-based coding artifacts, students and educators will
be empowered to learn about the rapidly-growing field of
AI, and gain skills necessary for the future of work. Peer Reviewed Paper
Evan W. Patton, Audrey Seo, and Franklyn Turbak,
(2019). Enhancing Abstraction in App Inventor with Generic
Event Handlers. Blocks & Beyond 2019.
Abstract: Work on code
smells in blocks languages has found that programmers often duplicate
blocks code rather than abstracting over common patterns of
computation using procedure-like features. For example, previous
analyses of over a million MIT App Inventor projects have revealed
that procedures are used surprisingly rarely in the wild and that many
users miss opportunities for using procedural abstraction to avoid
code duplication in their projects.
In this work, we use data analysis to explain how particular features
of App Inventor create barriers to abstracting over event handlers. In
many cases, duplicated code in event handlers cannot be extracted into
a procedure without using so-called generic blocks that abstract over
a particular component (e.g., a label). Generic blocks are rarely used
in practice, possibly because programmers do not know about them or
find them difficult to use. But even proceduralization with generic
blocks does not remove the need for duplicating the event handlers
We address these issues with two enhancements to App Inventor. First,
we add generic event handlers, a new form of abstraction that allows
specifying a single handler for all components of a particular
type. Second, we add a way to easily convert between specific and
generic blocks to facilitate genericization, that is,
abstracting actions over a particular component to apply to a group of
components of that type.
We also discuss related design choices and ways to encourage
programmers to use the new features to avoid code duplication. Our
work is an example of data-informed programming language design, in
which the creation or modification of features is informed by the
analysis of large datasets of programs from the language's target
audience. Peer Reviewed Paper
Evan W. Patton, (2019). Exploring Blockly
Performance: Case Studies in MIT App Inventor. Blockly
User Summit 2019 (Oct 10-11, 2019, Mountain View, CA).
Abstract: As the first project to use Blockly, the MIT App Inventor
users have pushed the boundaries of Blockly's capabilities. In our
datasets we have seen projects with upwards of 1400 blocks. I
present two ways in which we measure performance and four different
techniques used in App Inventor's production system to improve on
Blockly performance for large workspaces. These improvements include
reducing the amount of time spent in rendering and three different
improvements to worst case runtime for bulk operations on workspaces
with many blocks. Invited Talk
Evan W. Patton, (2019). Realtime Collaboration
with Blocky: Experiences with MIT App Inventor. Blockly
User Summit 2019 (Oct 10-11, 2019, Mountain View, CA).
Abstract: In her Masters of Engineering thesis, Deng explored three
different approaches to multi-user collaboration on MIT App Inventor
projects. I briefly introduce the different computational models and
network infrastructures for collaboration, and explain how we chose
the implementation we did for realtime collaboration in App
Inventor. A brief overview of Deng's user study and preliminary
results are also presented. Invited Talk
Danny Tang, Yuria Utsumi, and Natalie Lao, (2019). PIC: A Personal Image Classification Webtool for High School Students. IJCAI 2019 EduAI Workshop.
Abstract: As machine learning (ML) permeates our society
and manifests itself through commonplace tech-
nologies such as autonomous vehicles, facial recog-
nition, and online store recommendations, it is nec-
essary that the increasing number of people who
rely on these tools understand how they work. As
such, we need to develop effective tools and cur-
ricula for introducing ML to novices. Our work
focuses on teaching core ML concepts with image classification, one of the most basic and widespread
examples of machine learning. We built the PIC, a
web interface that allows users to train, test, and
analyze personalized image classification models
on pictures taken with their webcams, as well as a companion extension for MIT App Inventor that al-
lows users to use their models to classify objects in
a mobile application. We also created high school
level curricula for workshops based on the afore- mentioned interface and MIT App Inventor,
and ran pilot workshops with two classses of high school students from Boston Latin Academy.
Our findings indicate that high school students with no ML background are able to learn and understand gen- eral concepts and applications of ML through us- ing our tools, as well as successfully utilize models they build for personal use. Peer Reviewed Paper
Natalie Lao, Irene Lee, and Hal Abelson, (2019). A Deep Learning Practicum: Concepts and Practices for Teaching Actionable Machine Learning at the Tertiary Education Level.
ICERI 2019 Conference.
Abstract: Despite a rapid expansion of machine learning (ML) across fields and industries, little is known about
how to best prepare students for work in ML. Most ML courses today are taught at the college or
professional level using a theoretical programming approach. Existing educational resources may not
be sufficient or preferable for many audiences, particularly those who do not have a strong computer
science background, who wish to attain basic understanding of and ability to use ML. We present
learnings from a beginner level, semester-long actionable machine learning course taught at
Massachusetts Institute of Technology meant to be accessible for students with minimal computer
science knowledge. Based on analysis of survey responses and student projects, we find 5 core
concepts (Multilayer Networks, Convolutional Neural Networks, Transfer Learning, Recurrent Neural
Networks, and Embeddings & Generative Models) and 8 core skills (scoping a problem, choosing
datasets, creating datasets, choosing models, modifying models, creating models, modifying learning
rates, and training & testing) that helped lead to student self-efficacy as independent ML developers
when mastered. We conclude by discussing implications of this research on effective course design
and educational efforts for beginner level university courses in ML. Peer Reviewed Paper
Jessica Van Brummelen, (2019). Conversational Agents to Democratize Artificial Intelligence. 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).
Artificial Intelligence (AI) technology can be found nearly
everywhere. However, developing and controlling this tech-
nology is generally limited to large companies or those with
extensive training in computer science. It is difficult for many
who have application ideas for AI to even know where to start
when developing AI technology.
In , I empower students with little to no programming
experience to develop conversational AI agents (programs that
use AI and natural language to communicate with humans).
Through high school workshops and block-based coding tools,
students created agents to help people remember forgotten
words, determine how to recycle items, and learn math
concepts. Students learned computational thinking (CT) and
machine learning (ML) concepts through developing these
Nevertheless, this system has limitations. It uses the Alexa
Skills platform, which requires users to have access to Alexa
Developer and Amazon Web Services accounts. As well, to
speak to the agents, one must begin all conversations with
the word, ”Alexa”, which can be awkward and frustrating.
Furthermore, the visual coding tool limits its user-base to
those with regular vision and those who know how to use
a traditional computer. Conference Paper
Tissenbaum, M., Sheldon, J., & Abelson, H. (2019). From Computational Thinking to Computational Action. Communications of the ACM, March 2019, Vol. 62 No. 3, Pages 34-36.
Abstract: Computational action, a new framing for computing education, proposes that while learning about computing, young people should also have opportunities to create with computing that have direct impact on their lives and their communities. In this Viewpoint, we outline two key dimensions of computational actioncomputational identity and digital empowermentand further argue that by focusing on computational action in addition to computational thinking, we can make computing education more inclusive, motivating, and empowering for young learners. Learners have the capacity to develop computational products that can have authentic impact in their lives from the moment they begin learning to code, all they need is to be situated in contexts that allow them to do so.
Too often, K-12 computing education has been driven by an emphasis on kids learning the "fundamentals" of programming. Even more progressive CS education that centers around the development of learners' computational thinking has largely focused on learners understanding the nuanced elements of computation, such as variables, loops, conditionals, parallelism, operators, and data handling.10 This initial focus on the concepts and processes of computing, leaving real-world applications for "later" runs the risk of making learners feel that computing is not important for them to learn. It begs the question far too many math or physics students have asked, "When will we use this in our lives?"
See the article at CACM
Kevin Zhu, An Educational Approach to Machine Learning with Mobile Applications, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2019.
Abstract: Machine learning has increasingly become a major topic in computer science for stu- dents to learn. However, it can be quite technical and thus difficult for students to grasp, especially those in high school and under. To make machine learning and its applications more accessible to younger students, we developed a series of machine learning extensions for MIT App Inventor. MIT App Inventor is a web application for users with minimal programming experience to easily and quickly build mobile applications, and these extensions allow users to build applications that incorporate powerful machine learning functionality. These extensions were tested over a 6-week class with about 10 students and can be used as an educational tool.
Danny Tang, Empowering Novices to Understand and Use Machine Learning With Personalized Image Classification Models, Intuitive Analysis Tools, and MIT App Inventor, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2019.
Abstract: As machine learning permeates our society and manifests itself through commonplace technologies such as autonomous vehicles, facial recognition, and online store recom- mendations, it is necessary that the increasing number of people who rely on these tools understand how they work. As such, we need to develop effective tools and curricula for introducing machine learning to novices. My work focuses on teaching core machine learning concepts with image classification, one of the most basic and widespread examples of machine learning. I built a web interface that allows users to train and test personalized image classification models on pictures taken with their computers webcams. Furthermore, I built an extension for MIT App Inventor, a platform for building mobile applications using a blocks-based programming lan- guage, that allows users to use the models they built in the web interface to classify objects in their mobile applications. Finally, I created high school level curricula for workshops based on using the aforementioned interface and App Inventor extension, and ran the workshops with two classes of high school students from Boston Latin Academy. My findings indicate that high school students with no machine learning background are able to learn and understand general concepts and applications of machine learning through hands-on, non-technical activities, as well as successfully utilize models they built for personal use.
Kelsey K. Chan, Building an Online Community of Creators Through MIT App Inventor, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2019.
Abstract: An online community whose primary activity involves sharing personal creations is called an Online Community of Creators (OCOC). These communities provide a space where one users work can serve as inspiration for others and where users can offer feedback to each other, thus fostering creativity and helping users learn and grow. I designed and implemented a stand-alone gallery where App Inventor programmers can showcase their projects and receive feedback on their work. The goals of this new gallery were (1) to encourage more users to share their projects and (2) to increase the discoverability of shared projects among the App Inventor community. Based on the user study results, the new gallery addresses many usability pain points that users faced in the old gallery. Users expressed that they are more likely to share projects and view projects using the new gallery. By continuing to improve the usability of the gallery user interface, we hope to encourage even more users to publish projects to the gallery and to explore projects created by other members of the App Inventor community.
Jessica Van Brummelen Tools to Create and Democratize Conversational Artificial Intelligence, M.S. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2019.
Abstract: The world is becoming increasingly saturated with voice-first technology, such as Ama- zon Alexa and Google Home devices. As this technology becomes more complex, the skill set needed to develop conversational AI applications increases as well. This work bridges the gap, democratizes AI technology, and empowers technology consumers to become technology developers. In this thesis, I develop block-based Alexa Skill programming tools, enabling anyone even elementary school students to create complex conver- sational AI applications. During high school workshops, students created Alexa Skills to help others remember forgotten words, learn math concepts, ease recycling, and display Alexas speech on screen for those hard of hearing. Additionally, I developed a conversational AI curriculum and partnered with MITs High School Studies Program to provide workshops to the Boston community. We taught students about the capabilities, limitations, and implications of conversational AI, and explored research questions, such as "What do students believe, understand, and think about conversational agents?"; "Can students develop their own conversational AI applications?"; and "What do students envision for the future of conversational AI?" The results from a pre-workshop assessment suggested that despite not understand- ing how conversational agents worked, students could think of ways for conversational agents to solve problems. The post-workshop assessment suggested that through the workshops, students learned conversational AI and machine learning concepts; could identify capabilities and limitations of conversational agents; felt proud of their project development; were interested in developing their projects further; and were generally hopeful and excited about the future of conversational AI. Through this research, students learned about the power and limitations of AI, were empowered to solve real-world problems using AI, and developed socially useful conversational AI agent applications.
Nichole I. Clarke, Through the Screen and into the World: Augmented Reality Components with MIT App Inventor, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2019.
Abstract: Augmented reality (AR) is an immersive technology which augments the real-world with virtual content. The added, virtual content is grounded in the real-world, provides in-context information, and creates experiences that could not be perceived by ones senses alone.
As an educational technology, augmented reality has the power to motivate students and greatly shape their learning [1,2]. However, theres a lack of educational AR applications, and many educators and learners do not have the technical training to develop such applications.
I hypothesize that one solution is to empower students by providing them with direct access to AR development tools and grounding their learning through computational action. My work focuses on creating such augmented reality development tools, which are comprehensible and employable by anyone.
I built upon MIT App Inventor, a blocks-based mobile development tool, to democratize augmented reality application development. I developed a set of 20 augmented reality components, which allows users to build iOS augmented reality applications and enables the development of curriculum that promotes computational action with AR. I created a set of tutorials for the AR components and designed sample applications using those components with increasing levels of complexity. Finally, I conducted a pilot study with MIT students where they built applications using the augmented reality components.
My findings indicate that study participants felt empowered by working with the AR components. Participants, who originally doubted their ability to create AR applications, believed that they could build augmented reality applications after using the AR components.
Evan W. Patton, Michael Tissenbaum, and Farzeen Harunani (2019). MIT App Inventor: Objects, Design, and Development. In: Computational Thinking Education, S. C. Kong & H. Abelson (Eds.), Springer Nature, New York.
Abstract: MIT App Inventor is an online platform designed to teach computational
thinking concepts through development of mobile applications. Students create applications by dragging and dropping components into a design view and using a visual
blocks language to program application behavior. In this chapter, we discuss (1) the
history of the development of MIT App Inventor, (2) the project objectives of the
project and how they shape the design of the system, and (3) the processes MIT uses
to develop the platform and how they are informed by computational thinking literature. Key takeaways include use of components as abstractions, alignment of blocks
with student mental models, and the benefits of fast, iterative design on learning.
Mike Tissenbaum, Josh Sheldon, and Hal Abelson, (2018). Reducing the Barriers for Computational Action: How Blocks-Based Programming Can Support Digital Empowerment. In: Proceedings of the BLOCKS+ Workshop, co-located with SPLASH 2018 (Nov. 5, 2018, Boston, MA).
Abstract: Computational action, by which learners are given opportunities to build projects with real-world applicability, is, we argue, a preferable way to engage young people in learning how to program computers. Mobile computing, as the dominant form-factor of computing currently, provides an especially meaningful opportunity for youth to create these projects. Meanwhile, platforms with powerful abstractions, that make operations that might otherwise be difficult simple, build student capability to create projects that enhance their computational action.
Natalie Lao, (2018). CloudDB in MIT App Inventor. In: Proceedings of the BLOCKS+ Workshop, co-located with SPLASH 2018 (Nov. 5, 2018, Boston, MA).
Abstract: Data sharing through Cloud technology is one of the most powerful new computer science concepts of the past few decades. As such, developing powerful and easy-to-use tools for incremental learning and application of shared data concepts is an important endeavor. My work focuses on using MIT App Inventor, a popular blocks-based mobile application development tool for teaching computational thinking to young students, to make shared data technology understandable and usable by anyone without the need for extensive computer science training. I present the deployment of CloudDB, a set of coding blocks for MIT App Inventor that allows users to store, retrieve, and share various types of data in tag-value pairs on a Redis server for their mobile applications.
Evan W. Patton, Stefania Druga, Jessica Van Brummelen, Joe Mazzone, and Xavier Puig Fernandez, (2018). Panel: Block Abstractions for Artificial Intelligence. In: Proceedings of the BLOCKS+ Workshop, co-located with SPLASH 2018 (Nov. 5, 2018, Boston, MA).
Abstract: Block-based programming languages, such as Scratch and MIT App Inventor, make computational concepts accessible to those as young as primary school students. As technology advances, so do the abilities of such block-based programming languages. This panel will discuss the state-of-the-art in artificial intelligence (AI) abstractions in block-based languages. Specifically, it will address block-based AI challenges, pedagogical value, and applications.
Evan W. Patton, Mark Sherman, and Michael Tissenbaum, (2018). Research Tools for MIT App Inventor. In: Proceedings of the BLOCKS+ Workshop, co-located with SPLASH 2018 (Nov. 5, 2018, Boston, MA).
Abstract: Understanding how ideas move through a classroom as students explore, learn, and share their findings with other students is important for computational thinking education. We present a set of enhancements to MIT App Inventor to enable fine-grained analysis of both qualitative and quantitative data. Our framework combines real-time event streams, project snapshots, and screen captures with audio recording to allow for targeted queries around when components or concepts not previously seen appear in a project. This automated approach reduces the amount of time spent sifting through data by providing researchers new tools for performing analysis of App Inventor projects.
Evan W. Patton and Danny Tang, (2018). JSON Interoperability in MIT App Inventor. In: Proceedings of the BLOCKS+ Workshop, co-located with SPLASH 2018 (Nov. 5, 2018, Boston, MA).
Wen Xi and Evan W. Patton, (2018). A Blocks-Based Approach to Internet of Things in MIT App Inventor. In: Proceedings of the BLOCKS+ Workshop, co-located with SPLASH 2018 (Nov. 5, 2018, Boston, MA).
Abstract: Internet of Things (IoT) integrates physical devices and creates opportunities for people to interact with the surrounding
environment. While a number of blocks-based approaches
exist for programming some hardware, such as Snap! for
Arduino, Microblocks, and Scratch extensions, this is still an
underexplored area. In this paper, we propose a block-based
programming approach using MIT App Inventor to enable
novices be able to build mobile apps integrated with IoT
technology. We also review other block languages applied
for IoT. We conclude with some thoughts on how blocks
languages might inspire people to create with IoT.
J. Van Brummelen and H. Abelson. "'What's conversational AI?' with MIT App Inventor and Amazon Alexa", Amazon Research Days 2018, Boston, Oct. 2018.
Abstract: This work aims to democratize conversational AI technology. Currently, MIT App Inventors block-based coding tools empower anyone to develop their own mobile apps. Soon, users will also be able to program their own Alexa skills, conversational AI, and neural networks using blocks. Through middle school workshops, research questions, such as "Can students learn about the capabilities, limitations and implications of AI through developing conversational AI mobile apps?", will be explored.
S.C. Kong, D. Andone, G. Biswas, G., T. Crick, T., H.U. Hoppe, T. C. Hsu, R.H. Huang, K.Y. Li, C.K. Looi, M. Milrad, J. Sheldon, J.L. Shih, K.F. Sin, M. Tissenbaum, & J. Vahrenhold (Eds.),
On Tools that Support the Development of Computational Thinking Skills: Some Thoughts and Future Vision, Proceedings of the International Conference on Computational Thinking Education 2018. Hong Kong: The Education University of Hong Kong.
Abstract: Development of Computational Thinking (CT) is an area of many initiatives in the last years, due to the importance of having CT skills. There are many environments that allow learners to develop such skills, for instance Scratch and MIT App Inventor, in a visual and intuitive way. As in professional software development, assisting tools that help and guide learners are starting to appear. In this paper, we discuss the current status of these tools, based on an analysis of what state-of-the-art CT assessment tools, such as Dr. Scratch for Scratch and CodeMaster for App Inventor, offer. We report their limitations and envision and discuss future enhancements.
Peer Reviewed Paper
C.H. Tseng, M. Tissenbaum, W.H. Kuan, F.C. Hsu, C.C. Wong. A design-based approach to implementing a computational thinking curriculum with App Inventor and the Internet of Things, CTE 2018, July 2018
Abstract: The growing ubiquity of everyday devices connected over the internet, know generally as the Internet of Things (IoT), has opened up new avenues for students to explore their worlds and think and create computationally.
Combining IoT with mobile technologies (such as smartphones), enables students to move their designs and computational thinking out of traditional classroom settings and into the real world. This article outlines a design-based IoT curriculum that connects Taiwanese students with the personally-relevant issue of air pollution. The curriculum employs student-driven smartphone application design, using MIT's App Inventor, with Wi-Fi enabled IoT devices (LinkIt 7697
Wi-Fi/BLE MCU board). This paper reports on changes to the curriculum based on a preliminary pilot and observations of student engagement during the most recent enactment.
Peer Reviewed Paper
L. Tang. Constrained Sets: The Effects of Multi-Layered Environments in Learning App Inventor term paper, Massachusetts Institute of Technology, June 2018
Abstract: MIT App Inventor is a mobile application development platform that seeks to democratize the construction of mobile apps by making app development accessible to people with little to no experience with script-based programming. It uses block-based programming to introduce and teach programming concepts to its users. Users drag and drop functional and visual components onto their planned app in the screen editor, and construct the logic behind those components by using blocks in the block editor. In this thesis, we design and implement Constrained Sets, a system that allows instructors and developers to allow access to only a subset of App Inventor functionality by hiding component and block access. This system allows for the construction of multi-layered interfaces, which we then use to conduct an experiment that explores how novice App Inventor users learn App Inventor in different interface environments. Furthermore, we discuss and test the possibility of using a React based implementation of the App Inventor designer, and what implications that may have on creating more flexible user interfaces.
K. Prutz, Democratizing Cross-Platform Mobile Development with MIT App Inventor, term paper, Massachusetts Institute of Technology, Spring 2018.
Abstract: App Inventor is an intuitive, blocked-based programming environment that allows everyoneeven childrento build fully functional mobile apps for smartphones and tablets. App Inventor has been around for 8 years now, has users in almost every country in the world, and has 6.8 million users across the globe.
App Inventor democratizes mobile development through block-based coding, but because App Inventor targets only the Android operating system, this excludes the 20% of people worldwide who use Apple devices, including schools that are deploying iPads for use in the classroom. In this project, weve extended MIT App Inventor by natively developing components for iOS in Swift, allowing for the development of iOS applications through block-based programming. This provides users with the ability to develop Android and iOS versions of the same application, without any additional coding and without the need for platform-specific development hardware. Upon randomly sampling 1,831,065 existing App Inventor projects, 68.805% of the projects could be developed for both iOS and Android. This solution takes great steps toward democratizing cross-platform mobile development for Android and iOS applications.
Student Term Paper
K. Prutz, H. Abelson, Expanding Device Functionality For the MIT App Inventor IoT Embedded Companion, term paper, Massachusetts Institute of Technology, May 17, 2018.
Abstract: AppInventor allows anyone, even - and especially - children, to create fully functional apps in a quick and trouble-free way. AppInventor+IoT expands this to include communication between the device running the app and remote devices, most notably Arduinos. The next step with AppInventor+IoT is to allow for fully autonomous projects to run on the Arduino. This is what the ?App Inventor IoT Embedded Companion (hereafter Embedded Companion) makes possible. While this was in development, I ?created project tutorials to complement the addition of the EmbeddedCompanion into AppInventor+IoT. I made tutorials for two projects, the Autonomous Healthy Plant and the Puzzle Box, to showcase the power of autonomously-run projects. I also made slight adjustments to the existing how-to documents for setting up the Arduino and setting up a BLE connection between the app and the Arduino. Since the EmbeddedCompanion was in development while I was building the tutorials, I extended device functionality to meet the needs of these two projects: light sensor, humidity and temperature
sensor, moisture sensor, touch sensor, buzzer, and LCD.
Student Term Paper
K. Wang, Creating a Database Log History Page for Cloud Components in App Inventor to Support Software Development Education" term paper, Massachusetts Institute of Technology, 2018.
Abstract: CloudDB, an experimental feature of the MIT App Inventor Platform, allows users to send data across several different machines using tag (descriptor) and value (data) pairs. However, currently with CloudDB, users, by deploying several blocks, can only view specific values corresponding to known tags. Since App Inventor, a blocks-based app development interface, emphasizes the usability of its program, in this work I implement a web interface where users can view and search the entirety of their CloudDB data history. The user, for each separate project, can see their tags, the corresponding stored data, and a
history of that data. Detailed user studies were conducted to discover efficacy in teaching shared data concepts using this tool. Coupled with educational workshops designed for increasing computational thinking, this extension of CloudDB helps bolster student comprehension of advanced computing concepts and strengthen their independence in design, development, and debugging processes. This work demonstrates that a comprehensive visual tool appended
to a cloud database system can increase understanding and thinking about a technology so prevalent in many of todays technologies. Student Term Paper
M. Sherman, assisted by O. M. Alkhatib, Technovation Computational Thinking 2018, Massachusetts Institute of Technology, 2018.
Abstract: The Technovation Challenge is the largest technological entrepreneurship program for young
women in the world. The program is run by Iridescent, who has already collected evidence
that the challenge increases the rate of majoring in computer science by the girls in their
program. This is especially important, and women are consistently underrepresented in
Technovation requires a mobile app prototype, and around 80% of apps submitted this year
were built with MIT App Inventor. In this analysis, a team from MIT App Inventor took a closer
look at half of those projects. The MIT team broke down the projects into a number of
features and metrics, provided interpretation for the data patterns that were found, and
provided abstracted data representing these features so that Iridescent may conduct further
This analysis was conducted under an agreement between Iridescent and MIT App Inventor.
The mechanism of data collection, extraction, and analysis follows in the methodology.
Link to Analysis
M. Tissenbaum, J. Sheldon, L. Seop, C.H. Lee, N. Lao, Critical computational empowerment: Engaging youth as shapers of the digital future
Global Engineering Education Conference (EDUCON), 2017 IEEE, 1705-1708, April 2017
Abstract: While there is growing recognition of the need to support young learners as they develop computational thinking (CT) skills, this paper advocates for an increased focus on computational identity (CI) and digital empowerment (DE) in particular, as we posit that these CT skills will help young learners become self-motivated, innovative creators. When nurturing identity and empowerment, it is a persistent challenge to provide youth with real-world experiences and suitable development tools. This paper advances two key approaches to address these issues: 1) A critical computational literacy approach to engage youth in developing personally meaningful applications that have impact in the real world; 2) the use of a tool, like MIT App Inventor, that lowers barriers to creating useful, impactful technology. Using two case-based studies, we show how these two approaches have supported youth in developing applications that respond to meaningful challenges in their communities and helped them to establish their computational identities and digital empowerment.
Peer Reviewed Paper
S. Kong, H. Abelson, J. Sheldon, A. Lao, M. Tissenbaum, M. Lai, Curriculum Activities to Foster Primary School Students Computational Practices in Block-Based Programming Environments The Education University of Hong Kong, Hong Kong, 84 2017, CTE 2017
Abstract: As digital technology is increasingly a part of all sectors of society, educational approaches must be developed in order to nurture students ability to see the world through a computational lens. One way to achieve this goal is to promote Computational Thinking (CT) for young learners. The CoolThink@JC project is a four-year curriculum pilot designed to integrate CT into Hong Kong upper-primary level schools. The CoolThink framework for curriculum development is structured around computational concepts, practices and perspectives adapted from the framework of Brennan and Resnick (2012). This adapted framework motivated the choice of learning activities for CoolThink. This paper focuses on one aspect of that framework, namely computational practices. Here, we describe how activities in the CoolThink curriculum can promote the computational practices highlighted by the framework.
Peer Reviewed Paper
L. L. Ruan, E. W. Patton, and M. Tissenbaum, Evaluations of programming complexity in App Inventor, presented at CTE 2017, Hong Kong, Hong Kong, July 2017.
Abstract: To understand computational thinking in App Inventor, it is important to be able to effectively evaluate computational complexity in block-based programming languages. In the past, there have been a handful of complexity measures proposed for text-based languages (Weyuker, 1988). In this paper, we will attempt to implement 2 such measures, Halsteads Programming Effort and statement count, in App Inventor on a dataset of projects from 50 random users. The goal is to determine whether or not text programming standards for complexity can be generalized to block programming languages. This paper shows that the 2 complexity measures we implemented are not adequate measures for complexity in App Inventor. This result indicates a need for different measures of complexity that more accurately portray block programming proficiency. We hope this study will be a gateway into a better understanding of the intricacies of App Inventors block programming language and its unique contributions to the development of computational thinking.
Peer Reviewed Paper
D. Bau, J. Gray, C. Kelleher, J. Sheldon, and F. Turbak, Learnable programming: Blocks and beyond, Comm. of the ACM, vol. 60, no. 6, pp. 72-80, Jun, 2017.
Abstract: A global push to broaden participation in computer science has led to an explosion of interest in blocks-based programming. Visual blocks are used by numerous programming tools. Millions of students receive their first exposure to programming via these tools in courses and activities like Code.orgs Hour of Code. Blocks allow beginners to compose programs without struggling with the frustrations of syntax (Figure 1).
There is increasing interest in developing and studying blocks languages. At VL/HCC 2015, a small workshop session called Blocks and Beyond ballooned to a large event, with 51 submissions and 36 presenters. Researchers shared work in new blocks languages, interface innovations, domain-specific applications of blocks, and ways to make blocks languages more effective and accessible for diverse coders.
This article explores how blocks impact the learnability of programming. We begin by reviewing studies on the effectiveness of blocks languages. Then we discuss the key features of blocks languages and how they relate to learning. Finally, we look at applications of blocks in new domains and discuss tools for creating your own blocks language.
Peer Reviewed Paper
M. Tissenbaum, J. Sheldon, E. W. Patton, A. Gupta, E. Zhang, and D. Gopinath, Off the screen, and into the world of everyday objects: Computational thinking for youth with the Internet of Things, presented at CTE 2017, Hong Kong, Hong Kong, Jul, 2017.
Abstract: This paper discusses the opportunities presented by the growth of the Internet of Things (IoT) to provide youth opportunities to develop their computational thinking and digital empowerment. This paper argues that to support youth in developing these literacies, we need to develop platforms that reduce the barriers of entry while still allowing them to explore and develop their computational identities. To this end, this paper introduces an extension to App Inventor by MIT that enables students to quickly design, develop, and implement IoT applications. We outline one IoT activity for youth and future directions for both curricular and technical development.
Peer Reviewed Paper
N. Lao, "CloudDB: Components for Exploring Shared Data with MIT App Inventor" ,
Data sharing through Cloud technology is one of the most powerful new computer science concepts of the past few decades. As such, developing powerful and easy-to-use tools for incremental learning and application of shared data concepts is an important endeavor. My work focuses on using MIT App Inventor, a popular blocks-based mobile application development tool for teaching computational thinking to young students, to make shared data technology understandable and usable by anyone without the need for extensive computer science training. I present the ongoing development of CloudDB, a set of coding blocks for MIT App Inventor that allows users to store, retrieve, and share various types of data in tag-value pairs on a Redis server for their mobile applications.
Peer Reviewed Paper
N. Lao, Developing cloud and shared data capabilities to support primary school students in creating mobile applications that affect their communities, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2017.
Abstract: As technology and society become further intertwined, it is imperative that we democratize the creation of technology and educate people to be capable of harnessing the full power of computational thinking. As such, developing meaningful tools and curricula for incremental learning of computational thinking concepts starting in primary education is an important endeavor . My work focuses on making Cloud technology, one of the most powerful new computer science concepts, understandable and usable by anyone without the need for extensive computer science training.
I used MIT App Inventor, a blocks-based mobile application development tool for teaching computational thinking to young students, as the platform for my research. I developed CloudDB, a set of coding blocks for MIT App Inventor that allows users to store, retrieve, and share various types of data in tag-value pairs on a Redis server for their mobile applications. I created middle and high school level curricula based on CloudDB along with assessment tools to evaluate my materials and the extent to which young students can understand and utilize the concepts around shared data. Finally, I ran one of those workshops with middle school students in the MIT area.
My findings indicate that teaching shared data as a core computational thinking concept is entirely feasible to students as young as middle school level. Students are capable of inferring and extrapolating other use cases and potential problems with the Cloud, such as storage limits and security concerns. When given the context of solving a problem in their lives, they are very driven and able to design and create complex independent mobile application projects using MIT App Inventor and CloudDB.
X. Deng, Group collaboration with App Inventor, term paper, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2017.
Abstract: Collaboration becomes increasingly important in programming as projects become more complex. With traditional text-based programming languages, programmers typically use a source code management system to manage the code, merge code from multiple authors, and optionally lock files for conflict-free editing. There is a limited corpus of work around collaborative editing of code in visual programming languages such as block-based programming. I present a collaborative programming environment to MIT App Inventor, a web-based visual platform for building Android applications with blocks, which enables many programmers to collaborate in real time on the same MIT App Inventor project. I design and implement three collaboration models to evaluate the efficacy of these different collaborative models for users of App Inventor so as to understand which approach best enables collaboration. Our results demonstrate that real-time programming decreases the completion time of a task, improves the interaction between users, and increases users likeability towards collaborative programming. I anticipate that this new collaborative programming environment will change the way users use MIT App Inventor, and more curriculum based on the collaborative tools will be designed.
L. Tang, Creating customized user interfaces for App Inventor, term paper, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2017.
Abstract: Many of App Inventors users pick up App Inventor outside of a formal learning environment by going through online walkthroughs and tutorials. By using a block-based programming language, App Inventor democratizes technology by lowering the bar of entry for people around the world. For my thesis, I plan to create and test a feature that allows for the formation of customized component and block sets in App Inventor.
Student Term Paper
X. Deng, E.W. Patton, (2017) Enabling Multi-User Computational Thinking with Collaborative Blocks Programming in MIT App Inventor. In: Proceedings of the 1st International Conference on Computational Thinking in Education (July 13-15, 2017, Hong Kong, China).
Collaboration becomes increasingly important in programming as projects become more complex. With traditional text-based programming languages, programmers typically use a source code management system to manage the code, merge code from multiple editors, and optionally lock files for conflict-free editing. There is a limited corpus of work around collaborative editing of code in visual programming languages such as block-based programming. We propose an extension to MIT App Inventor, a web-based visual platform for
building Android applications with blocks, which will enable many programmers to collaborate in real-time on MIT App Inventor projects. We take the position that real-time collaboration within MIT App Inventor will encourage students in a group environment to interact with one another in ways that help them improve each others understanding and practice of computational thinking practices that may not be achieved in the traditional one user-one project paradigm that is currently provided.
Peer-reviewed Conference Paper
E. W. Patton, X. Deng, (2017). Collaborative Mobile App Development with MIT App Inventor. In: Proceedings of Blocks & Beyond 2017 (Oct. 10, 2017, Raleigh, NC).
Abstract: MIT App Inventor has been used around the world to introduce students to explore computational thinking in the context of designing and building mobile applications. Currently, App Inventor has a limitation of one user per project. This makes group work in an educational setting difficult because it creates an artificial barrier to collaborative work. The rise of cloud-based, realtime systems to collaborate on other types of creative works, such as Google Docs, suggests that there is a gap between the production capabilities of App Inventor and what users might expect from online tools.
In this presentation, we will demonstrate an extension to App Inventor that allows multiple users to simultaneously edit shared projects in both the designer and blocks editors. Actions are synchronized between workspaces in near realtime, allowing for a "Google Docs"-like experience for group work on App Inventor projects. User awareness is enhanced through colored highlights to indicate the presence and focus of other users in the shared workspace. The demonstration will include hands-on use of the realtime system.
G. Campbell, Extending cloud data store component for MIT App Inventor, term paper, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2017.
Abstract: MIT App Inventor required a new cloud storage component to provide cloud sharing capabilities for Android applications built with App Inventor. I helped design and implement such a component, named CloudDB, which allows App Inventor users to store application data and media files on an MIT hosted Redis server. The component initially provided key-value storage and basic list operations. I improved it by adding advanced list operations, image, sound and other media file storage, as well as ensuring that data stored on the server would be transmitted securely to and from user devices via SSL.
Student Term Paper
E. Giurleo, Keyboard interactions in block-based coding environments, term paper, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2017.
Abstract: App Inventor is a website that allows users to make their own Android apps using a block-based coding language. While App Inventor has almost 3 million users, it contains few accessibility features. In particular, there is no way for a user to navigate the block workspace on the App Inventor site without using their mouse; this poses a significant problem for users who use screen readers or who cannot operate a mouse. I have designed and implemented a set of keyboard shortcuts that allow users to operate the App Inventor website in an intuitive way. I have also conducted a usability study to determine the effectiveness of this keyboard-accessible interface. Based on the results of the study, I conclude that much of the interface is intuitive to users with experience using the keyboard in other domains, but that there are changes that can be made to make the design even more intuitive.
Student Term Paper
J. Im, User-defined blocks for MIT App Inventor, term paper, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2017.
Abstract: MIT App Inventor is an open source web platform that lets users build their own Android Apps quickly by using blocks-based programming. MIT App Inventor currently has diverse types of components, from general blocks to specific task oriented components like sensors. However, it does not provide every type of block that an user needs. For example, currently there are no components for intricate graph visualization in App Inventor. However, there is a limitation to keep adding new categories of blocks, not only because the complexity of maintaining the system will increase, but also because an individual users need is necessarily not a general need for others. In order to keep lowering the barriers of creating special goal oriented apps and to foster the learning of computational thinking, while also preventing the situation of adding every possible block to the system, a new feature called user-defined blocks (customizable blocks) has been implemented during 2017 Spring semester.
Student Term Paper
W. Li, Helping the helpers: A Toolkit for mobile humanitarian assistance apps, M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2016.
Abstract: My research investigates the following question:? ?How can relief workers be empowered to create useful mobile apps to support the work of first responder teams? Mobile devices are reshaping the disaster management domain and they make fast and targeted support possible. However there is the major obstacle of relief workers and volunteers often lacking the technical abilities to build and deploy the right mobile app in response to a particular disaster. In addition, data interoperability is often missing in many of the applications since they were developed without any prior agreement on the data schema. Linked Data technology solves the data interoperability problem by defining a method of publishing structured data and these structured data can be interlinked and become more useful. In order to explore and provide a solution to the gap between the knowledge of the relief workers and volunteers and the technical abilities needed to
create an app, I conducted a participatory design workshop with the people at the International Committee of Red Cross and developed two mobile applications with one of the project managers there. In addition, I have created a DoItYourself (DIY) toolkit that includes: (1) a Mobile Linked Data App Kit: a streamlined process for creating mobile apps and (2) an appbuilding methodology: a set of principles for relief workers to follow while creating apps using the toolkit. The goal of this toolkit is to enable aid workers to create vital and necessary mobile apps and to empower the humanitarian community. User testing of the framework with first responders has shown that the idea of using, modifying, and creating an app is greatly favored by our
participants, that mobile Linked Data applications can aid humanitarian organizations by increasing their impact and effectiveness, and that humanitarian workers can successfully develop
and deploy mobile applications by using this toolkit.
A. Colter, Evaluating and improving the usability of MIT App Inventor,
M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2016.
Abstract: MIT App Inventor is a blocks-based programming language for Android apps designed to teach programming skills to middle school and high school students. We aim to make app development accessible for all. Anyone learning to program must learn computational thinking methods; App Inventor users must also learn how to use the service. Our target users, teenagers and people without programming experience, often conflate the two learning processes: they think App Inventor is hard because learning to program is hard. As such, App Inventor needs a
user interface that matches the conventions of commercially- available software our users already know how to use. Such an interface will allow them to focus on learning how to program and to transfer their knowledge and skills to other programming languages and environments.
I designed several tasks and conducted a usability study on the existing, publicly- available App Inventor service. Users encountered 75 unique issues and a total of 157 issues. This is an average of 5 unique issues and 10.5 total issues per user. I made changes to the App Inventor source code that addressed 34 of the most common issues encountered. My intent was to make App Inventor both more usable to novice programmers and more similar to the programming environments that experienced programmers use. Finally, I conducted a usability
study with the same tasks on the modified version of App Inventor. Users encountered 65 unique issues, including 19 issues encountered in the first study, and 107 total issues. Based on user
comments and behavior, I conclude that my solutions resolved 21 of the original issues, partially mitigated 9 issues, and did not improve the usability of 4 issues.
B. Xie, Progression of computational thinking skills demonstrated by App Inventor users, M.Eng thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2016.
Abstract: I analyze skill progression in MIT App Inventor, an open, online learning environment
with over 4.7 million users and 14.9 million projects/apps created. My objective is to
understand how people learn computational thinking concepts while creating mobile
applications with App Inventor. In particular, I am interested in the relationship
between the development of sophistication in using App Inventor functionality and the
development of sophistication in using computational thinking concepts as learners
create more apps. I take steps towards this objective by modeling the demonstrated
sophistication of a user along two dimensions: breadth and depth of capability. Given
a sample of 10,571 random users who have each created at least 20 projects, I analyze
the relationship between demonstrating domain-specific skills by using App Inventor
functionality and generalizable skills by using computational thinking concepts.
I cluster similar users and compare differences in using computational concepts.
My findings indicate a common pattern of expanding breadth of capability by
using new skills over the first 10 projects, then developing depth of capability by
using previously introduced skills to build more sophisticated apps. From analyzing
the clustered users, I order computational concepts by perceived complexity. This
concept complexity measure is relative to how users interact with components. I
also identify differences in learning computational concepts using App Inventor when
compared to learning with a text-based programming language such as Java. In
particular, statements (produce action) and expressions (produce value) are separate
blocks because they have different connections with other blocks in App Inventors
visual programming language. This may result in different perceptions of computational concepts when compared to perceptions from using a text-based programming language, as statements are used more frequently in App Inventor than expressions.
This work has implications to enable future computer science curriculum to better
leverage App Inventors blocks-based programming language and events-based model
to offer more personalized guidance and learning resources to those who learn App
Inventor without an instructor.
S. Tsai, Understanding App Inventor forums, M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2016.
Abstract: MIT App Inventor is a visual blocks language that allows users to create mobile applications for Android. App Inventor users have the option of posting in a public support forum to discuss anything from specific programming issues to education is- sues. In order to leverage the information on the forums to improve App Inventor, we must first understand what is being discussed. In this thesis, we used unsupervised machine learning methods to automate discovery of discussion topics. First, we transformed posts into feature vectors using a bag-of-words model. Next, we clustered posts using k-means clustering and evaluated our results both quantitatively, by calculating the average silhouette of the posts, and qualitatively, by simply looking at the clusters of posts. Finally, we used LDA topic modeling to determine the topics being discussed and compared the extracted topic words to a manual evaluation of each cluster. Using this technique, we were able to uncover common problems with App Inventor that users encountered. We hope to use this information to improve users experience with App Inventor.
K. Asmus, A system of applications for the integration of BLE beacons in museums, M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2016.
Abstract: There is great potential for BLE beacon technology to be incorporated into museum exhibits to provide more enriching content to visitors and gather feedback to improve the museum overall. I developed a system of three applications to accomplish these goals: one for museum staff to easily deploy and update beacons, another for visitors to engage with the supplemental material transmitted by beacons at various exhibits, and a third for museum staff to understand visitors interactions and feedback provided by the beacons.
I implemented two versions of the system according to two different beacon protocols: Eddystone- UID, which broadcasts to a dedicated mobile app; and Eddystone-URL, which broadcasts to mobile browsers via the Physical Web.
Each system was tested in a real-world context through experiments hosted by the MIT Museum. I analyzed the success and potential of such systems based on observations, participant responses, and data gathered via the system during the experiments. The reliability of BLE connections remains an issue, and beacon content only appealed to a small subset of visitors; however, those who interacted with the system valued the experience highly and the MIT Museum expressed interest in providing a richer experience for these visitors.
B. Xie, I. Shabir, and H. Abelson, Measuring the usability and capability of App Inventor to create mobile applications, in PROMOTO 2015, Pittsburgh, PA, 2015, p. 108.
Abstract: MIT App Inventor is a web service that enables users with little to no previous programming experience to create mobile applications using a visual blocks language. We analyze a sample of 5,228 random projects from the corpus of 9.7 million and group projects by functionality. We then use the number of unique blocks in projects as a metric to better understand the usability and realized capability of using App Inventor to implement specific functionalities. We introduce the notion of a usability score and our results indicate that introductory tutorials heavily influence the usability of App Inventor to implement particular functionalities. Our findings suggest that the sequential nature of App Inventor's learning resources results in users realizing only a portion of App Inventor's capabilities and propose improvements to these learning resources that are transferable to other programming environments and tools.
Peer Reviewed Paper
R. Morelli, C. Uche, P. Lake, & L. Baldwin, Analyzing year one of a CS principles project, in 46th ACM Technical Symposium on Computer Science Education, Kansas City, MO, 2015, pp. 368-373.
Abstract: The Mobile Computer Science Principles (Mobile CSP) project is an NSF-funded CS 10K project. Its goal is to train a new cohort of high school computer science teachers to teach an Advanced Placement (AP) level course based on the emerging Computer Science Principles (CSP) framework. Mobile CSP uses App Inventor, a graphical programming language for Android devices, to engage students in app building as
a means to get them interested in computer science. This paper reports on the first year of this effort. In addition to describing the projects Professional Development (PD) course, a 6-week, full-time summer course for teachers, and the Mobile CSP curriculum, the paper provides a preliminary analysis of demographic and performance data obtained from high school students who took the course during the 2013-2014 academic year.
Peer Reviewed Paper
F. Turbak, M. Sherman, F. Martin, D. Wolber, and S. Crawford Pokress, Events-first programming in App Inventor,Journal of Computing Sciences in Colleges, vol. 29, no. 6, Jun, 2014, pp 81-89.
Abstract: Events are a central concept in computer science. They are becoming more important with the prevalence of mobile and web platforms that use event-based programming. Yet, events are typically taught late in the CS curriculum -- e.g., in a web programming or operating systems course. We have introduced events to CS0 students from day one using MIT App Inventor, a blocks-based programming environment that enables students to create apps for Android devices. This paper presents the system's event-based model, along with typical coding problems and best-practice approaches for solving them. We advocate for increasing early emphasis on events in the CS curriculum.
Peer Reviewed Paper
D. Y. Miao, PrivacyInformer: An automated privacy description generator for the MIT App Inventor, M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2014.
Abstract: With the Advent of smart mobile phones and ubiquitous mobile applications, the pace at which people generate, access, and acquire data has accelerated significantly. In this thesis, we first examine how privacy concerns in the mobile apps market compromise the well-being of both app consumers and developers. Then, we study the problem with the lack of usable privacy documents in mobile apps. Subsequently, we propose a technical solution named PrivacyInformer, as an extension to the MIT App Inventor, a do-it-yourself mobile app building platform. This platform has a vast international user base, as well as a growing impact on the democratizing of mobile app building. We show that by analyzing the source code of mobile apps, PrivacyInformer can produce simple and useful privacy descriptions in both human-readable and machine-readable format. Finally, we conduct an exploratory user study to evaluate the effectiveness of PrivacyInformer from the app developer's perspective, as well as discuss the policy impact of such a tool in the mobile app development community.
D. Wolber, H. Abelson, and M. Friedman, Democratizing computing with App Inventor, Mobile Computing and Communications Review, vol. 18, no. 4, pp 53-58, Oct. 2014.
Abstract: MIT App Inventor is a visual blocks language that enables beginners and non-programmers to create apps for their phones and tablets. It has empowered thousands to create software with real-world usefulness, and see themselves as creators rather than only consumers in the mobile computing environment. Educationally, it offers a gateway drug that can help broaden and diversify participation in computing education.
Peer Reviewed Paper
J. Schiller, F. Turbak, M. Friedman, H. Abelson, J. Okerlund, J. Dominguez, and A. McKinney, Live programming of mobile apps in App Inventor, in Programming for Mobile and Touch (PRoMoTo), Portland, OR, 2014, pp. 1-8.
Abstract: MIT App Inventor is a programming environment that lowers the barriers to creating mobile apps for Android devices, especially for people with little or no programming experience. App Inventor apps for a mobile device are constructed by arranging components with a WYSIWYG editor in a computer web browser, where the development computer is connected to the device by WiFi or USB. The behavior of the components is specified using a blocks-based graphical programming language. A key feature in making App Inventor accessible to beginning programmers is live programming: Developers interact directly with the state of the evolving program as it is being constructed, and changes made in the web browser are realized instantaneously in the running app on the device. This paper describes the live programming features of App Inventor and explains how they are implemented.
Peer Reviewed Paper
F. Shih and J. Boortz, Understanding peoples preferences for disclosing contextual information to smartphone apps, in International Conference on Human Aspects of Information Security, Privacy, and Trust (HAS 2013), Las Vegas, NV, 2014, pp 186-196.
Abstract: Smartphones have become the primary and most intimate computing devices that people rely on for their daily tasks. Sensor-based and network technologies have turned smartphones into a context-aware information hub and a vehicle for information exchange. These information provide apps and third party with a wealth of sensitive information to mine and profile user behavior. However, the Orwellian implications created by context-awareness technology have caused uneasiness to people when using smartphone applications and reluctance of using them . To mitigate peoples privacy concerns, previous research suggests giving controls to people on how their information should be collected, accessed and shared. However, deciding who (people or the application) gets to access to what (types of information) could be an unattainable task. In order to develop appropriate applications and privacy policies it is important to understand under what circumstances people are willing to disclose information.
Peer Reviewed Paper
F. Zhang, F. Shih and D. Weitzner, No surprises: Measuring intrusiveness of smartphone applications by detecting objective context deviations,in ACM CCS Workshop on Privacy in the Electronic Society (WPES 2013), Berlin, Germany, 2013, pp 291-296.
Abstract: We address the challenge of improving transparency for smartphone applications by creating tools that assesses privacy risk. Specifically, we invented a framework for qualitatively assessing and quantitatively measuring the intrusiveness of smartphone applications based on their data access behaviors. Our framework has two essential components. The first component is the Privacy Fingerprint, a novel visualization that is concise yet holistic. It captures each app's unique access patterns to sensitive personal data, including which types of behaviors and under which privacy-relevant usage contexts the data are collected. The second component is a new Intrusiveness Score that numerically measures out-of-context data collection, based on real data accesses gathered from empirical testing on 33 popular Android apps across 4 app categories. Specific attention is paid to the proportion of data accesses that occurs while the user is idle, raising the perceived level of intrusiveness and exposing the profiling potential of an app. Together, these components will help smartphone users decide whether to install an app because they will be able to easily and accurately assess the relative intrusiveness of apps. Our study also demonstrates that the Intrusiveness Score is helpful to compare apps that exhibit similar types of data accesses.
Peer Reviewed Paper
A.K. Saigal, Information accountability for mobile financial applications, M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2010.
Abstract: In this thesis, I designed and built three sets of applications for three different demographics - young people, elderly people, and people in the developing world - to enable them to be involved in their personal banking. Members of these demographics are not actively involved in their personal banking when compared to others. We believe that part of the discrepancy lies in the lack of convenience and accountability. Thus, we have developed applications, whereby the issues of convenience and accountability are addressed. The applications are built around mobile devices, which will likely make them accessible since members of these demographics are often on mobile devices. The applications are also built around rules that can be set by a responsible party, so that the users know exactly what can and cannot be done with their money (such as a father restricting the amount of money his son can withdraw, or what the money can be used for). Finally, we keep a history of every transaction, who initiated it, and an explanation given by the initiator so we can understand why it occurred. Using our applications, built around convenience and accountability, will allow banks to reach youth, elderly people, and people in the developing world in ways that they have not been able to previously.
F. Shih, O. Seneviratne, D. Miao, I. Liccardi, L. Kagal, E. W. Patton, P. Meier, and C. Castillo, Democratizing mobile app development for disaster management, in AIIP Workshop on Artificial Intelligence Problems and Approaches for Intelligent Environments and Workshop on Semantic Cities, Beijing, China, 2013, pp 39-42.
Abstract: Smartphones are being used for a wide range of activities including messaging, social networking, calendar and contact management as well as loca- tion and context-aware applications. The ubiquity of handheld computing technology has been found to be especially useful in disaster management and relief operations. Our focus is to enable developers to quickly deploy applications that take advantage of key sources that are fundamental for todays net- worked citizens, including Twitter feeds, Facebook posts, current news releases, and government data. These applications will also have the capability of empowering citizens involved in crisis situations to contribute via crowdsourcing, and to communicate up-to-date information to others. We will leverage several technologies to develop this application framework, namely (i) Linked Data principles for structured data, (ii) existing data sources and ontologies for disaster management, and (iii) App Inventor, which is a mobile application development framework for non-programmers. In this paper, we describe our motivating use cases, our architecture, and our prototype implementation.
Peer Reviewed Paper
R. Morelli, D. Wolber, S. Pokress, F. Turbak, and F. Martin, Teaching the CS principles curriculum with App Inventor (abstract only), in Special Interest Group on Computer Science Education 44th Association for Computing Machinery Technical Symposium on Computer science education, Denver, CO, 2013, pp. 762-762.
Abstract: The CS Principles Project is an NSF-funded initiative to develop a breadth-first advanced placement (AP) course in computer science. App Inventor is a visual, blocks-based programming language that makes sophisticated computing concepts accessible to a broad range of students. This hands-on workshop, aimed at high school and undergraduate teachers, will introduce participants to lessons, homework exercises, project assignments, and assessment materials (quizzes, grading rubrics) that can be used in an App Inventor-based CS0 course. Participants will develop simple Android apps, using devices provided by the workshop, and will use them in the context of lessons and assignments that fit within the CS Principles framework.
Peer Reviewed Paper
MS. Crawford Pokress and J. J. Dominguez Veiga, MIT App Inventor: Enabling personal mobile computing, presented at Programming for Mobile and Touch (PRoMoTo), Indianapolis, IN, 2013.
Abstract: MIT App Inventor is a drag-and-drop visual programming tool for designing and building fully functional mobile apps for Android. App Inventor promotes a new era of personal mobile computing in which people are empowered to design, create, and use personally meaningful mobile technology solutions for their daily lives, in endlessly unique situations. App Inventor's intuitive programming metaphor and incremental development capabilities allow the developer to focus on the logic for programming an app rather than the syntax of the coding language, fostering digital literacy for all. Since it was moved from Google to MIT, a number of improvements have been added, and research projects are underway.
Peer Reviewed Paper
F. Shih and W. Li, Unlocking smartphone data for educational use in teaching and learning environment, presented at the Smart University Workshop in Eighth International and Interdisciplinary Conf. on Modeling and Using Context, Annecy, France, 2013.
Abstract: We address the challenges of applying smartphone data to improve the experiences of teaching and learning in the university environment. Specifically, we created a framework to collect, monitor and analyze student behavioral data and increase
social interactions for educational uses. The framework has three essential components. The first component is App Inventor, a web-based platform with drag-and-drop tools for novices to create mobile apps that collect personal and environmental data. The second component is Reactive Data Store, an intuitive mechanism that connects multiple data sources and automates the tasks for analyzing personal data. The third component is a push notification mechanism that delivers personalized or group messages as
the results of the real-time analysis and triggers more actions accordingly. Together, these components will transform people's campus experiences by unleashing the values of smartphone data. Lastly, we introduced the framework to a group of teachers in
higher education with a focus group discussion to identify the plausible use cases in terms of performance assessments, pedagogy design and collaborative learning. Peer Reviewed Paper
Hsu, Y. -C., & Ching, Y. -H. (2013). Mobile app design for teaching and learning: Educators experiences in an online graduate course. The International Review of Research in Open and Distance Learning, 14(4), 117-139.
This research explored how educators with limited programming experiences learned to design mobile apps through peer support and instructor guidance. Educators were positive about the sense of community in this online course. They also considered App Inventor a great web-based visual programming tool for developing useful and fully functioning mobile apps. They had great sense of empowerment through developing unique apps by using App Inventor. They felt their own design work and creative problem solving were inspired by the customized mobile apps shared by peers. The learning activities, including sharing customized apps, providing peer feedback, composing design proposals, and keeping design journals (blogging), complemented each other to support a positive sense of community and form a strong virtual community of learning mobile app design. This study helped reveal the educational value of mobile app design activities and the web-based visual programming tool, and the possibility of teaching/learning mobile app design online. The findings can also encourage educators to explore and experiment on the potential of incorporating these design learning activities in their respective settings, and to develop mobile apps for their diverse needs in teaching and learning. Peer Reviewed Paper
A. Bhagi, Android game development with App Inventor, M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2012.
Abstract: Ever since App Inventor found a new home at MIT's Center for Mobile Learning in early 2012 (it was previously housed at Google), it has been receiving significant attention from educational communities that hope to use the tool to facilitate technology learning in the classroom and beyond.
Much of App Inventor's current user base is teachers in high-schools and colleges, and as the tool continues to attract more users, it is likely that many of these users will be teachers of students in grades 3 thru 12 (i.e. students who are old enough to know how to use computers and design/develop apps, yet not experienced enough to get their hands dirty with the Android SDK). Students of this age group are quite fond of gaming, and so we anticipate that there will be a strong demand for developing graphically appealing, interactive single-player and multiplayer games using App Inventor in the coming years. It is therefore important that the App Inventor team at MIT prepare for the imminent growth of the App Inventor game development community. Accordingly, this thesis looks at where App Inventor currently stands with respect to game development and how its game development capabilities can be improved and extended. Accepted Thesis
F. Turbak, S. Sandu, O. Kotsopoulos, E. Erdman, E. Davis and K. Chadha, Blocks languages for creating tangible artifacts, in Institute of Electrical and Electronics Engineers Symposium on Visual Languages and Human-Centric Computing, Innsbruck, Tyrol, Austria, 2012, pp 137-144.
Abstract: Logo turtles and Hendersons picture language have long been used to teach computational thinking by inspiring learners to construct programs that
create complex geometric designs. We have developed visual blocks-based versions of these languages, TurtleBlocks and PictureBlocks, that allow users to transform
their designs into tangible artifacts produced by laser cutters and vinyl cutters. Our languages embody two novel features. First, they use constructive area geometry to
convert the geometric designs generated by our programs into formats suitable for laser and vinyl cutters. Second, they leverage static typing and polymorphism to provide
a new way to reference the names of procedure parameters and local variables in a blocks language. Peer Reviewed Paper
Lauren E. Margulieux, Mark Guzdial, and Richard Catrambone, Subgoal-labeled instructional material improves performance and transfer in learning to develop mobile applications, Proceedings of ICER 2012, p 71-78. Mental models are mental representations of how an action changes a problem state. Creating a mental model early in the learning process is a strong predictor of success in computer science classes. One major problem in computer science education, however, is that novices have difficulty creating mental models perhaps because of the cognitive overload caused by traditional teaching methods. The present study employed subgoal-labeled instructional materials to promote the creation of mental models when teaching novices to program in Android App Inventor. Utilizing this and other well-established educational tools, such as scaffolding, to reduce cognitive load in computer science education improved the performance of participants on novel tasks when learning to develop mobile applications. Peer Reviewed Paper
A.K. Saigal and A. Saigal, Saathimobile and the rapid deployment of prototypes to build applications for social enterprise in the developing world, in 1st International Conference on Wireless Technologies for Humanitarian Relief, Amritapuri, Kollam, Keralla, India, 2011, pp. 351-356.
Abstract:This paper describes a sales, marketing, and inventory management application, SaathiMobile, for the distribution of low-cost sanitary napkins, made out of waste banana fiber,
to women in rural areas. It is being pilot tested in Hubli, Karnataka. This product has the potential to have great impact on the health and economic welfare of women in these areas.
The product was developed using App Inventor, a rapid prototype deployment environment for Android. Peer Reviewed Paper
B. Magnuson, "Building blocks for mobile games: A multiplayer framework for App Inventor for Android," M.Eng. thesis, Elect. Eng. Comput. Sci., Massachusetts Inst. of Technol., Cambridge, 2010.
Abstract: Building Blocks for Mobile Games is a client-server multiplayer game building framework for the App Inventor for Android platform. The framework includes an App Inventor component and a game server running on Google App Engine. The client side component packages the complexity of web service calls, data transfer and game state management into a set of graphical code blocks that allow users without programming experience to create Android applications that can access the game server API. The default game server provides basic functionality that can be used to create simple multiplayer games and message passing applications, such as a multi user bulletin board. By using custom server modules, the game server can be enhanced to provide server commands that implement a wide range of operations. Custom modules were used with the system to develop a multi- player card game, a version of Bulls and Cows with a shared scoreboard, an application that accesses Amazons book search API and a pair of applications for creating, managing and voting in polls. Using App Inventor and the game framework, each of these applications can be created with only a basic understanding of mobile phones and client-server principles. Accepted Thesis