Active learning is a set of techniques that require the student to take an active role in their learning during lecture. Research strongly supports that students will learn more when the lecture utilizes these techniques. And I have measured this effect in my own courses. However, this research shows that students like lectures that use these techniques less even though they are learning more. And I have also informally measured this, such as students who say at the end of the first lecture, "If you are going to require me to participate in lecture, I will not return". Unfortunately, the present educational model is based on the student evaluations (primarily measuring what students like) to evaluate the quality of instruction. Therefore perversely, this aggregate model encourages suboptimal teaching and learning.
The paper recommends then that professors take time in the beginning of the semester to demonstrate the benefits and gain buy in from the students. And then continue to do so. Students want to learn, so they will support this pedagogy. And many students will recognize the value with time, if they give it.
Showing posts with label education. Show all posts
Showing posts with label education. Show all posts
Saturday, September 28, 2019
Wednesday, October 17, 2018
Thesis Defense: Practical Concurrency Testing
Ben Blum defended his dissertation work today on Practical Concurrency Testing. What follows are the notes from that defense.
To prove that a program is correct across arbitrary concurrency. There are three testing approaches:
unit testing of the most likely, stress testing that is not systematic, and verification that requires separate tools and techniques to describe.
Landslide is a proposed technique that is based on Stateless Model Checking (Godefroid '97), which tests a different execution interleaving on every iteration. However, the naive interleaving provides O(2^n) states to test. [Flanagan '05] identified equivalent interleavings and [Musuvathi '08] proposed heuristic orderings to identify the possible bugs faster. This approach can often require annotations, so adoption requires automated instrumentation. This space is addressing further concurrency problems such as weak memory models, but hardware transactional memory is still open.
This instrumentation requires preemption points. Finer-grained finds more bugs, but increases the states to test. Bugs / failures follow certain cases, such as use-after-free, deadlocks, assertion failures, and invalid memory accesses. Dynamic data-race analysis can help inform the necessary preemption points.
As a reminder, a data race:
- one or more accesses is write
- threads are not holding the same mutex
- Nor is there other ordering requirements (condition variable, etc)
Quicksand applies this analysis to select different smaller problem spaces using subsets of possible preemption points. Each subset also represents smaller parts of the larger possible problem space. If these subsets are all satisfied, then represents a full verification of the program. Prior work explored using APIs such as mutex_lock/unlock, or using every shared variable access as preemption points.
This tester is deployed in OS courses at CMU, PSU, and U Chicago. Manual annotation is not viable for students, especially those struggling for whom the traces would be valuable. That said, students regularly deploy ad-hoc synchronization, such as while (!ready) yield();, requires heuristics as the naive model checking must test every possible count of yielding and its interleaving.
When used by students, about 75% of tested kernels / libraries have identifiable bugs from the testing framework. For the tested submissions (7 semesters) of students at CMU, there is an improvement in grades, but it is not statistically significant when correcting for the opt-in bias. Most students are then able to fix their bugs found by the tool.
Hardware transactional memory poses a separate challenge for model checking. Aborted transactions are observationally equivalent to an immediately failed transaction. Furthermore, all transactions must be assumed to abortable, as there are many possible causes of aborts. As prior posts covered, this fact requires that any transaction have a valid abort path. And this abort path requires most of the verification.
Testing Landslide using hand-written tests, transactional data structures, and a TSX-based spinlock. Each set of tests has a concurrency or performance bug in the implementations. What about demonstrating that there are no bugs in implementation? With 10 hours of CPU time, verification is only possible for small cases on complex code. That said, practical testing so far only requires <4 preemptions to create the buggy scenario. There can be other bugs requiring an increasingly complex ordering, but generally those are very rare.
Abstraction reduction [Simsa '13], works to reduce primitives within implementations to verified components, such as mutual exclusion, etc. Using this technique then allows Landslide to verify the complex HTM implementations at higher thread counts.
In attendance are the recent instructors of Operating Systems and the TAs.
To prove that a program is correct across arbitrary concurrency. There are three testing approaches:
unit testing of the most likely, stress testing that is not systematic, and verification that requires separate tools and techniques to describe.
Landslide is a proposed technique that is based on Stateless Model Checking (Godefroid '97), which tests a different execution interleaving on every iteration. However, the naive interleaving provides O(2^n) states to test. [Flanagan '05] identified equivalent interleavings and [Musuvathi '08] proposed heuristic orderings to identify the possible bugs faster. This approach can often require annotations, so adoption requires automated instrumentation. This space is addressing further concurrency problems such as weak memory models, but hardware transactional memory is still open.
This instrumentation requires preemption points. Finer-grained finds more bugs, but increases the states to test. Bugs / failures follow certain cases, such as use-after-free, deadlocks, assertion failures, and invalid memory accesses. Dynamic data-race analysis can help inform the necessary preemption points.
As a reminder, a data race:
- one or more accesses is write
- threads are not holding the same mutex
- Nor is there other ordering requirements (condition variable, etc)
Quicksand applies this analysis to select different smaller problem spaces using subsets of possible preemption points. Each subset also represents smaller parts of the larger possible problem space. If these subsets are all satisfied, then represents a full verification of the program. Prior work explored using APIs such as mutex_lock/unlock, or using every shared variable access as preemption points.
This tester is deployed in OS courses at CMU, PSU, and U Chicago. Manual annotation is not viable for students, especially those struggling for whom the traces would be valuable. That said, students regularly deploy ad-hoc synchronization, such as while (!ready) yield();, requires heuristics as the naive model checking must test every possible count of yielding and its interleaving.
When used by students, about 75% of tested kernels / libraries have identifiable bugs from the testing framework. For the tested submissions (7 semesters) of students at CMU, there is an improvement in grades, but it is not statistically significant when correcting for the opt-in bias. Most students are then able to fix their bugs found by the tool.
Hardware transactional memory poses a separate challenge for model checking. Aborted transactions are observationally equivalent to an immediately failed transaction. Furthermore, all transactions must be assumed to abortable, as there are many possible causes of aborts. As prior posts covered, this fact requires that any transaction have a valid abort path. And this abort path requires most of the verification.
Testing Landslide using hand-written tests, transactional data structures, and a TSX-based spinlock. Each set of tests has a concurrency or performance bug in the implementations. What about demonstrating that there are no bugs in implementation? With 10 hours of CPU time, verification is only possible for small cases on complex code. That said, practical testing so far only requires <4 preemptions to create the buggy scenario. There can be other bugs requiring an increasingly complex ordering, but generally those are very rare.
Abstraction reduction [Simsa '13], works to reduce primitives within implementations to verified components, such as mutual exclusion, etc. Using this technique then allows Landslide to verify the complex HTM implementations at higher thread counts.
In attendance are the recent instructors of Operating Systems and the TAs.
Monday, February 20, 2017
Repost: Learn by Doing
I want to take a brief time to link to two of Mark Guzdial's recent posts. Both including an important theme in teaching. Students learn best by doing not hearing. Oddly students commonly repeat this misconception. If I structure our class time to place them as the ones doing something, rather than me "teaching" by speaking, the appraisal can be that I did not teach. They may not dispute that they learned, but I failed to teach them.
Students learn when they do, not just hear. And Learning in MOOCs does not take this requirement into account.
I have to regularly review these points. So much so that I was able to give them to a group of reporters last week (part of new faculty orientation, but still).
Students learn when they do, not just hear. And Learning in MOOCs does not take this requirement into account.
I have to regularly review these points. So much so that I was able to give them to a group of reporters last week (part of new faculty orientation, but still).
Friday, October 14, 2016
Conference Attendance Teaching and Learning Summit 2016 - Keynote
Critical Thinking: Why is it so hard to teach? - Dr Daniel T. Willingham
Critical thinking is intertwined with content knowledge. We saw a sequence of four examples (If vowel then even number, if alcohol then 21, if gin, then haddock, if entering then cholera vaccine), for each example, there is a claim about a set of cards: If X then Y. Given four cards, verify the claim. If the problems were formulated based on permissions, then the success rate was high. Each problem is technically, P -> Q, but having just completed a semester of logic has no impact on results.
Scientific reasoning is taught in two pieces scientific concepts and scientific method. So consider designing a learning experiment. The group is split into intervention and control. How do you know that the random sample is valid? Background knowledge is required to determine the appropriateness of the split.
Critical thinking occurs from learning at the deep level. The surface story is say, "tumors and rays". The deep question is whether it is modus pones, Netwon's third law, etc? However, memory is focused on the surface facts. Recall is based on those components.
Why not teach the deep structure immediately? Abstractions are hard to understand. Instead, learners have to see lots of surface structures all overlaying the same deep structure.
Sometimes failures in critical thinking are actually failures in basic knowledge. Furthermore, there are also innate biases, such as words refer to objects and attributes, and the world is full of agents and purposes.
Takeaway 1: Most of critical thinking is domain-specific.
Takeaway 2: In each domain, faculty should identify what they consider the important critical thinking skills.
Takeaway 3: Select content with an eye toward teaching these skills. Teach the critical thinking in the context of the content.
Takeaway 4: Critical thinking is a curricular issue. These skills require more than 1 semester to acquire.
Takeaway 5: Certain foundational concepts may run counter to the mind's biases. Students have functional knowledge that has worked so far. For example, "equals sign means put answer here".
Q. Translating domain skills in interdisciplinary work?
A. Don't know. Probably needing to know enough of the skills in the home domain to be able explore the other domain.
Q. If critical thinking is domain specific, how specific are domains?
A. Domains are nested. Proper application requires domain knowledge. Moving from cognitive psychology to social leaves [the speaker] less skilled, but still better than average. Into clinical psychology, they have a common basis, but limited ability to apply.
Friday, August 28, 2015
Repost: Incentivizing Active Learning in the Computer Science Classroom
Studies have shown that using active learning techniques improve student learning and engagement. Anecdotally, students have brought up these points to me from my use of such techniques. I even published at SIGCSE a study on using active learning, between undergraduate and graduate students. This study brought up an interesting point, that I will return to shortly, that undergraduate students prefer these techniques more than graduate students.
Mark Guzdial, far more senior than me, recently challenged Georgia Tech (where we both are) to incentivize the adoption of active learning. One of his recent blog posts lists the pushback he received, Active Learning in Computer Science. Personally, as someone who cares about the quality of my teaching, I support these efforts although I do not get to vote.
Faculty members at R1 institutions, such as Georgia Tech, primarily spend their time with research; however, they are not research scientists and therefore they are being called upon to teach. And so you would expect that they would do this well. In meeting with faculty candidates, there was one who expressed that the candidate's mission as a faculty member would be to create new superstar researchers. Classes were irrelevant to this candidate as a student, therefore there would be no need to teach well as this highest end (telos) of research justifies the sole focus on students who succeed despite their instruction, just like the candidate did. Mark's blog post suggests that one day Georgia Tech or other institutions may be sued for this sub-par teaching.
What about engagement? I (along with many students and faculty) attended a visiting speaker talk earlier this week and was able to pay attention to the hour long talk even though it was effectively a lecture. And for this audience, it was a good talk. The audience then has the meta-takeaway that lectures can be engaging, after all we paid attention. But we are experts in this subject! Furthermore, for most of us there, this is our subfield of Computer Science. Of course we find it interesting, we have repeatedly chosen to study it.
For us, the material we teach has become self-evidently interesting. I return to the undergraduate and graduate students that I taught. Which group is closer to being experts? Who has more experience learning despite the teaching? Who prefered me to just lecture? And in the end, both groups learned the material better.
Edit: I am by no means condemning all of the teaching at R1's or even Georgia Tech. There are many who teach and work on teaching well. The Dean of the College of Computing has also put some emphasis on this through teaching evaluations. Mark's post was partially noting that teaching evaluations are not enough, we can and should do more.
Mark Guzdial, far more senior than me, recently challenged Georgia Tech (where we both are) to incentivize the adoption of active learning. One of his recent blog posts lists the pushback he received, Active Learning in Computer Science. Personally, as someone who cares about the quality of my teaching, I support these efforts although I do not get to vote.
Faculty members at R1 institutions, such as Georgia Tech, primarily spend their time with research; however, they are not research scientists and therefore they are being called upon to teach. And so you would expect that they would do this well. In meeting with faculty candidates, there was one who expressed that the candidate's mission as a faculty member would be to create new superstar researchers. Classes were irrelevant to this candidate as a student, therefore there would be no need to teach well as this highest end (telos) of research justifies the sole focus on students who succeed despite their instruction, just like the candidate did. Mark's blog post suggests that one day Georgia Tech or other institutions may be sued for this sub-par teaching.
What about engagement? I (along with many students and faculty) attended a visiting speaker talk earlier this week and was able to pay attention to the hour long talk even though it was effectively a lecture. And for this audience, it was a good talk. The audience then has the meta-takeaway that lectures can be engaging, after all we paid attention. But we are experts in this subject! Furthermore, for most of us there, this is our subfield of Computer Science. Of course we find it interesting, we have repeatedly chosen to study it.
For us, the material we teach has become self-evidently interesting. I return to the undergraduate and graduate students that I taught. Which group is closer to being experts? Who has more experience learning despite the teaching? Who prefered me to just lecture? And in the end, both groups learned the material better.
Edit: I am by no means condemning all of the teaching at R1's or even Georgia Tech. There are many who teach and work on teaching well. The Dean of the College of Computing has also put some emphasis on this through teaching evaluations. Mark's post was partially noting that teaching evaluations are not enough, we can and should do more.
Monday, August 17, 2015
Course Design Series (Post 2 of N): Choosing a Textbook
Having now read both Programming Language Pragmatics, Third Edition and Concepts of Programming Languages (11th Edition), I have settled on the former as my textbook for the fall. I do not find either book ideally suited, and I wish that the Fourth Edition was being released this summer and not in November, which is why it still hasn't arrived.
For the choice of which textbook to use, it was "Concepts" to lose and having 11 editions, the text should also be better revised. I dislike reading the examples in a book and questioning how the code would compile. Beyond which, the examples felt quaint and contrived.
(edit) For example, I was reviewing the material on F# and copied in an example from the text:
let rec factorial x =
if x <= 1 then 1
else n * factorial(n-1)
Does anyone else notice that the function parameter is x on the first two lines and n on the last?!
Before the Concepts book is written off entirely, there are many valuable aspects. I enjoyed reading about the history of programming languages, especially for exposing me to Plankalkül. The work also took a valuable track in the subject by regularly looking at the trade-offs between different designs and features. This point certainly helped inform my teaching of the material.
Fundamentally, when I looked at the price of the two textbooks, the benefits of using the newer Concepts textbook could not outweigh the nearly doubled pricetag. Most of the positive points are small things and can be covered as addendums to the material.
(FCC note - Concepts of Programming Languages was provided free to me by the publisher.)
For the choice of which textbook to use, it was "Concepts" to lose and having 11 editions, the text should also be better revised. I dislike reading the examples in a book and questioning how the code would compile. Beyond which, the examples felt quaint and contrived.
(edit) For example, I was reviewing the material on F# and copied in an example from the text:
let rec factorial x =
if x <= 1 then 1
else n * factorial(n-1)
Does anyone else notice that the function parameter is x on the first two lines and n on the last?!
Before the Concepts book is written off entirely, there are many valuable aspects. I enjoyed reading about the history of programming languages, especially for exposing me to Plankalkül. The work also took a valuable track in the subject by regularly looking at the trade-offs between different designs and features. This point certainly helped inform my teaching of the material.
Fundamentally, when I looked at the price of the two textbooks, the benefits of using the newer Concepts textbook could not outweigh the nearly doubled pricetag. Most of the positive points are small things and can be covered as addendums to the material.
(FCC note - Concepts of Programming Languages was provided free to me by the publisher.)
Thursday, April 2, 2015
Course Design Series (Post 0 of N): A New Course
This fall I will again be Instructor of Record. My appointment is to teach CS 4392 - Programming Languages. This is an unusual situation in that the course has not been taught for over 5 years (last time was Fall 2009). Effectively, the course will have to be designed afresh.
The first step in course design (following L. Dee Fink and McKeachie's Teaching Tips: Chapter 2) is to write the learning objectives using the course description, along with prerequisites and courses that require this one. Let's review what we have:
Course Description: none
Prerequisites:
Undergraduate Semester level CS 2340 (which has the following description)
Object-oriented programming methods for dealing with large programs. Focus on quality processes, effective debugging techniques, and testing to assure a quality product.
Courses depending on this: none
Alright. Now I will turn to the CS curriculum at Georgia Tech. Georgia Tech uses a concept they call "threads", which are sets of related courses. CS 4392 is specifically in the Systems and Architecture thread. This provides several related courses:
CS 4240 - Compilers, Interpreters, and Program Analyzers
Study of techniques for the design and implementation of compilers, interpreters, and program analyzers, with consideration of the particular characteristics of widely used programming languages.
CS 6241 - Compiler Design
Design and implementation of modern compilers, focusing upon optimization and code generation.
CS 6390 - Programming Languages
Design, structure, and goals of programming languages. Object-oriented, logic, functional, and traditional languages. Semantic models. Parallel programming languages.
Finally, the ACM has provided guidelines for the CS curriculum. Not only does this provide possible options for what material I should include, but they have also provided several ACM exemplar courses (c.f., http://www.cs.rochester.edu/
To summarize, if my first step is to write the learning objectives, then I am on step 0: write the course description. In a couple of weeks, I plan on finishing my initial review of potential textbooks as well as the other materials covered above. That will provide me the groundwork for the description and then objectives.
The first step in course design (following L. Dee Fink and McKeachie's Teaching Tips: Chapter 2) is to write the learning objectives using the course description, along with prerequisites and courses that require this one. Let's review what we have:
Course Description: none
Prerequisites:
Undergraduate Semester level CS 2340 (which has the following description)
Object-oriented programming methods for dealing with large programs. Focus on quality processes, effective debugging techniques, and testing to assure a quality product.
Courses depending on this: none
Alright. Now I will turn to the CS curriculum at Georgia Tech. Georgia Tech uses a concept they call "threads", which are sets of related courses. CS 4392 is specifically in the Systems and Architecture thread. This provides several related courses:
CS 4240 - Compilers, Interpreters, and Program Analyzers
Study of techniques for the design and implementation of compilers, interpreters, and program analyzers, with consideration of the particular characteristics of widely used programming languages.
CS 6241 - Compiler Design
Design and implementation of modern compilers, focusing upon optimization and code generation.
CS 6390 - Programming Languages
Design, structure, and goals of programming languages. Object-oriented, logic, functional, and traditional languages. Semantic models. Parallel programming languages.
Finally, the ACM has provided guidelines for the CS curriculum. Not only does this provide possible options for what material I should include, but they have also provided several ACM exemplar courses (c.f., http://www.cs.rochester.edu/
To summarize, if my first step is to write the learning objectives, then I am on step 0: write the course description. In a couple of weeks, I plan on finishing my initial review of potential textbooks as well as the other materials covered above. That will provide me the groundwork for the description and then objectives.
Saturday, March 7, 2015
Conference Attendance SIGCSE 2015 - Day 2 / 3
I recognize
that Day 1 afternoon went “missing”. I
presented my poster and that consumed the sum total of my time. While I am happy with all that I achieved with my poster (writing IRB protocol, independent work, analyzing my teaching, et cetera), it was not considered as a finalist for the student research competition (SRC). Yet I received significant feedback and a number of follow-ons that I will have to try to evaluate the next time(s) I teach. I have been doing an excellent job of networking and speaking with my colleagues. And I have seen several exciting techniques to improve my teaching.
In traveling, take some time to prepare students. Let them know what to expect. For example, it is okay to miss some paper sessions, and even return to your room entirely. It is okay to ask questions 1:1. Find groups where people are being introduced and join in. Student volunteering, while takes time, also gives an additional individuals that you will know. Use the people you know to introduce you to others at the conference.
This is just what it sounds. A Ruby based framework that enables writing simple unit tests that will then be applied to a full simulation of the assembly executed.
The presenter(s) were not at this poster, but it showed a high quality interface for seeing the scheduling of threads according to different scheduling policies. The intent here was not to explore races and parallelism, but rather see how scheduling decisions are made in an OS.
I was not expecting this poster. You are walking along and then see 4 Raspberry Pi's all networked together. Raspberry Pis and HPC?! A small setup, but it is an interesting development that takes advantage of the low cost Pi and still provide an HPC platform for students.
Plastic parts all worked together to form replicas of Pascal's mechanical calculator. Interesting and student assembled.
Teams of 4
students, approach is evaluated on courses from three years of major (sophomore
on up). Teams are formed with CATME
(particularly using dissimilar GPAs in a group), as well as partner selection
(when possible). Students provide peer
evaluations after each stage of the project.
Significant data collection looking particularly at what students prefer
for to be the evaluation policy (between 100% of grade for the group’s work to
100% of the grade for the individual’s contribution). This question was taken repeatedly throughout
the semester, which leads to whether student preferences change? More senior students prefer more weight being
attributed to group. The predictor for
what grade split is at what point in the course is this surveyed, and effectively
as soon as the teams are formed the students prefer to be graded primarily as a
group. Follow on study is looking at
experience with team projects, trust in the ability to evaluate individual
contribution, and other questions. This
is a hopeful data point.
How do faculty become aware and why do they try out teaching practices? 66 participants in CS, including chairs, tenure-track faculty, teaching faculty, and Ph.D. student instructors across 36 institutions. First, the mental model of what an instructor does can differ significantly from what the instructor is actually doing. Second, faculty can find out about practices through a variety of approaches, such as self-identifying that there is possible improvement in their teaching. Faculty often trust other faculty like them (researchers to researches, lecturers to lecturers). Third, when adopting a practice, faculty need to evaluate the effectiveness (see also my poster, student feedback, etc). -- My efforts in this have been having different faculty (my recommendation letter writers) view my lectures / teaching, and thereby giving them demonstrations of different practices.
"We lost the war on cheating" Instead, we have to meet with students such that they are demonstrating their understanding of the code. The requirements of submissions: attribute your sources and understand your submission. Enables students to work together, use all sources, develop interview skills. Enables reuse of assignments. Grading is now 40% correctness / 60% code interview. Rubric for each interview. Students should arrive early and have their laptop ready to present / explain. Students were better able to learn and complete the assignments, as well as feedback for improvement. Students also felt better able to learn the material by being able to collaborate and not constrained by a collaboration policy. There are some stressors, such as TAs having to meet with hundreds of students, as well as their inconsistencies. -- This was perhaps the most exciting new technique that I saw / heard about.
Thursday, March 5, 2015
Conference Attendance SIGCSE 2015 - Day 1 Morning
It is colder here in Kansas City. Fortunately, I will only be outside briefly. Most often I will be networking and continuing my efforts to both become a better teacher, as well as finding an academic job teaching.
This morning, I am focusing on the "Curriculum" track. I am excited by the three papers in this track, the first looks at research, the second is on systems courses, and the last on parallel computing courses. Alas, I was in the hallway track and missed the first work. Perhaps I can find the authors later.
Backward Design: An Integrated Approach to a Systems Curriculum
The goal of systems is "higher level software creation". Computer Science courses are split into Core Tier 1 and Tier 2 (a term from the ACM 2013 curriculum), where the former are taken by all CS majors and the later are only taken by most or some. One issue in the old curriculum was that OS also taught C. In crafting a new curriculum, first establish a vision statement, which can be used in conflict resolution (and also revised). Establish SMART objectives to prepare and build the assessments. The results can be found on github.
A Module-based Approach to Adopting the 2013 ACM Curricular Recommendations on Parallel Computing
Parallel computing is important and important for CS graduates to know. The 2013 ACM Curriculum increased the number of hours that students should take in parallel computing. Part of the recommendations are to place parallel computing into the curriculum and not just as a course. Thus parallelism modules are placed throughout the curriculum (perhaps as early as CS1 or CS2). Find the level of abstraction for a concept and introduce it appropriately. For example, Amdahl's Law in CS1 versus cache coherence in senior-level class. 5 modules of parallelism were established, which have equivalences with the ACM. Each course in the curriculum may have 1 or more modules, which then teaches and reinforces the topics. Even after adding these modules, there has continued to be incremental development and revisions, which have improved student outcomes. The key take away is that it is possible to introduce these recommendations without completely rewriting the curriculum.
In the afternoon, I will be standing with my poster - Using Active Learning Techniques in Mixed Undergraduate / Graduate Courses. Later I will post updates from my afternoon.
This morning, I am focusing on the "Curriculum" track. I am excited by the three papers in this track, the first looks at research, the second is on systems courses, and the last on parallel computing courses. Alas, I was in the hallway track and missed the first work. Perhaps I can find the authors later.
Backward Design: An Integrated Approach to a Systems Curriculum
The goal of systems is "higher level software creation". Computer Science courses are split into Core Tier 1 and Tier 2 (a term from the ACM 2013 curriculum), where the former are taken by all CS majors and the later are only taken by most or some. One issue in the old curriculum was that OS also taught C. In crafting a new curriculum, first establish a vision statement, which can be used in conflict resolution (and also revised). Establish SMART objectives to prepare and build the assessments. The results can be found on github.
A Module-based Approach to Adopting the 2013 ACM Curricular Recommendations on Parallel Computing
Parallel computing is important and important for CS graduates to know. The 2013 ACM Curriculum increased the number of hours that students should take in parallel computing. Part of the recommendations are to place parallel computing into the curriculum and not just as a course. Thus parallelism modules are placed throughout the curriculum (perhaps as early as CS1 or CS2). Find the level of abstraction for a concept and introduce it appropriately. For example, Amdahl's Law in CS1 versus cache coherence in senior-level class. 5 modules of parallelism were established, which have equivalences with the ACM. Each course in the curriculum may have 1 or more modules, which then teaches and reinforces the topics. Even after adding these modules, there has continued to be incremental development and revisions, which have improved student outcomes. The key take away is that it is possible to introduce these recommendations without completely rewriting the curriculum.
In the afternoon, I will be standing with my poster - Using Active Learning Techniques in Mixed Undergraduate / Graduate Courses. Later I will post updates from my afternoon.
Monday, December 8, 2014
Improving Computer Science Education
Recently in the news are two articles that relate to improving Computer Science Education. It is valuable to broaden the base of students who understand the basics of computing, just as students are expected to know Chemistry or Calculus. In fact in my biased opinion, knowing the basics of computing and programming will have greater practical benefit to students; however, it should never be at the expense of a diverse education.
The White House announced that the 7 largest school districts will be including Computer Science in their curriculum. This will quickly lead to another problem of who will teach the Computer Science classes. Not me, but I am interested in teaching the teachers. I do want to see Computer Science as an actual specialty (endorsement) for Education majors.
Another aspect of broadening the base is retaining students enrolled in the major. Being part of the majority, it is difficult for me to know the challenges faced by other groups. Similarly, I know why I entered Computer Science, so I would like to understand why others have too. Why are they passionate or interested in this field that I am a part of? Here are some things minority students have to say about STEM.
The White House announced that the 7 largest school districts will be including Computer Science in their curriculum. This will quickly lead to another problem of who will teach the Computer Science classes. Not me, but I am interested in teaching the teachers. I do want to see Computer Science as an actual specialty (endorsement) for Education majors.
Another aspect of broadening the base is retaining students enrolled in the major. Being part of the majority, it is difficult for me to know the challenges faced by other groups. Similarly, I know why I entered Computer Science, so I would like to understand why others have too. Why are they passionate or interested in this field that I am a part of? Here are some things minority students have to say about STEM.
Monday, October 27, 2014
Liberal Arts College Positions Mini-Seminar
In continuing my search and preparation for a faculty position, today I attended a mini-seminar by LADO faculty. (LADO - liberal arts colleges with diversity officers) Here are some points that were raised during the breakout and panel discussions:
Teaching:
- You will teach both upper-level courses, as well as "service" courses. Service is a good term to describe the low-level / introductory courses, as the faculty are rotating through this service to the department.
- Try to setup the teaching schedule so that 1 day is free solely from research.
- Continue to revise courses so they are fresh and current, but also avoid constantly creating all new courses.
- Valuable to set aside "non-office hours" times, during which the door can be shut.
- Faculty will sit in on courses and additionally interview the students, as part of composing an evaluation of your teaching.
Research:
- Recruiting undergraduates earlier for research to have time to train them, so that they will later be able to contribute.
- You can still collaborate and have broad impact through research with faculty at other more research-focused institutions.
- Grant proposals can also be keyed "RUI" (research at undergraduate institutions)
- Regular funding for sabbatical leaves, first often after the renewal of the 3-year contract. This leave is focused on research and may be held at R1 or other institutions.
- Startup package is present to cover the transition to grant-based funding.
- Research lab costs are significantly lower at these institutions, as funds are not required for grad students, post docs, etc.
- Schools are looking for faculty hires that add diversity to the research available.
Service:
- Service is a component, but is much smaller than teaching and scholarship. So be cautious about accepting invitations to committees, in terms of time commitment. The service time can provide valuable insight to the functioning of the institution, as well as possible collaboration with collegues in other departments.
- You will be chair of your department someday.
Other:
- Many liberal arts institutions are located in small towns.
- Take the time to customize the cover letter. Do you really want and care about this job?
Teaching:
- You will teach both upper-level courses, as well as "service" courses. Service is a good term to describe the low-level / introductory courses, as the faculty are rotating through this service to the department.
- Try to setup the teaching schedule so that 1 day is free solely from research.
- Continue to revise courses so they are fresh and current, but also avoid constantly creating all new courses.
- Valuable to set aside "non-office hours" times, during which the door can be shut.
- Faculty will sit in on courses and additionally interview the students, as part of composing an evaluation of your teaching.
Research:
- Recruiting undergraduates earlier for research to have time to train them, so that they will later be able to contribute.
- You can still collaborate and have broad impact through research with faculty at other more research-focused institutions.
- Grant proposals can also be keyed "RUI" (research at undergraduate institutions)
- Regular funding for sabbatical leaves, first often after the renewal of the 3-year contract. This leave is focused on research and may be held at R1 or other institutions.
- Startup package is present to cover the transition to grant-based funding.
- Research lab costs are significantly lower at these institutions, as funds are not required for grad students, post docs, etc.
- Schools are looking for faculty hires that add diversity to the research available.
Service:
- Service is a component, but is much smaller than teaching and scholarship. So be cautious about accepting invitations to committees, in terms of time commitment. The service time can provide valuable insight to the functioning of the institution, as well as possible collaboration with collegues in other departments.
- You will be chair of your department someday.
Other:
- Many liberal arts institutions are located in small towns.
- Take the time to customize the cover letter. Do you really want and care about this job?
Saturday, March 8, 2014
Conference Attendance SIGCSE 2014 - Day 3
Today the day will be in reverse. We'll start with papers and end with the invited speaker. I have met many attendees and even talked to some of them. Let's start with operating systems and programming languages. With the bonus theme of avoiding using Linux for presentations.
Teaching OS through code review. Unified grading workflow with git, the student submissions are viewed as diffs and the grading is via online code review. Most students preferred this system over past solutions and tools. The system also supported incremental reviews / checkpoints. The GradeBoard tool is built on review board and git.
Virtual graphics card in qemu for teaching device driver design. Graphics is selected such that students would clearly see the results. Providing a device through a virtual machine significantly reduced the difficulties for instructors as well as for students. Minimal time required to restore student "machines" when they break. Most students completed the project versus earlier versions based on kernel intercepts.
A programming language compiler compiler. Earlier versions of the class require teaching scheme before students could implement their interpreter / compiler. Now based on java, the tool plcc processes provided lexical and grammar files, so that students can then interface with the java classes. Plcc only supports LL1 languages. Students implement simple interpreted languages.
And then it was time to network again, i.e. the hallway session. This continues to be an interesting expense for an introvert, yet it is also the exponential networking exercise. After I know more people, then it is more likely that I find a group in which that I know someone and can meet others. I've made progress with knowing the participants in my "field". And having more inspiration for teaching is summer.
Teaching OS through code review. Unified grading workflow with git, the student submissions are viewed as diffs and the grading is via online code review. Most students preferred this system over past solutions and tools. The system also supported incremental reviews / checkpoints. The GradeBoard tool is built on review board and git.
Virtual graphics card in qemu for teaching device driver design. Graphics is selected such that students would clearly see the results. Providing a device through a virtual machine significantly reduced the difficulties for instructors as well as for students. Minimal time required to restore student "machines" when they break. Most students completed the project versus earlier versions based on kernel intercepts.
A programming language compiler compiler. Earlier versions of the class require teaching scheme before students could implement their interpreter / compiler. Now based on java, the tool plcc processes provided lexical and grammar files, so that students can then interface with the java classes. Plcc only supports LL1 languages. Students implement simple interpreted languages.
And then it was time to network again, i.e. the hallway session. This continues to be an interesting expense for an introvert, yet it is also the exponential networking exercise. After I know more people, then it is more likely that I find a group in which that I know someone and can meet others. I've made progress with knowing the participants in my "field". And having more inspiration for teaching is summer.
Friday, March 7, 2014
Conference Attendance SIGCSE 2014 - Day 2
Well rested, it is time for conference again! Keynote today by code.org. When teaching someone programming, you don't tell them this is overloading or this is event handling, but instead the base concept. CS is starting to count for high school graduation requirements at the state level, but districts and universities are slower to change. CSEd Week is Dec 8 - 14 this year, for another Hour of Code. The first one had impressive results (including almost 50/50 male / female ratio) and the key thing is that this hour is providing a foot in the door. So the hour is meant as a just a start and 97% of teachers rated the hour positively.
Adding parallel programming in CS2. Students are taught OpenMP pragmas as applied to for loops. Projects assigned around matrix operations and image processing. Part of the teaching is done through live coding, which is based on demoing patternlets. Students see this component as exciting and fresh. All problems are restricted to those not requiring synchronization. (see Csinparallel.org).
Board game strategy development in CS2. Instructors provide the engine, which provides the graphics and true game state. Students write a player that maintains its representation of the state and decides on a move. In my CS3? we had a similar project with reversei / othello as the game. Then for research, half of the students were assigned to develop components in the game engine and other students developed the players. Students developing players had higher enjoyment and felt they learned more, although there was little difference in grades.
I also visited several posters that were interesting. In one, they studied why students dropped out of CS1 courses. Only two measures were statistically significant: first, how much computer science experience a student had before taking the class, and second, how busy (total work, not just credits) the student was that semester. Switching to active-learning had no real effect. Gender made no difference. Intention of majoring in computer science was not a factor.
The other poster looked at measuring the style of the code in CS1 assignments automatically. They found that their tool was able to cluster the student submissions based on stylistic similarity and that grades for each cluster had a 90% confidence. I'm intrigued! Style is important and being able to emphasize style further is great.
And then I talked with other attendees for many hours, which is one of the reasons that I'm there.
Adding parallel programming in CS2. Students are taught OpenMP pragmas as applied to for loops. Projects assigned around matrix operations and image processing. Part of the teaching is done through live coding, which is based on demoing patternlets. Students see this component as exciting and fresh. All problems are restricted to those not requiring synchronization. (see Csinparallel.org).
Board game strategy development in CS2. Instructors provide the engine, which provides the graphics and true game state. Students write a player that maintains its representation of the state and decides on a move. In my CS3? we had a similar project with reversei / othello as the game. Then for research, half of the students were assigned to develop components in the game engine and other students developed the players. Students developing players had higher enjoyment and felt they learned more, although there was little difference in grades.
I also visited several posters that were interesting. In one, they studied why students dropped out of CS1 courses. Only two measures were statistically significant: first, how much computer science experience a student had before taking the class, and second, how busy (total work, not just credits) the student was that semester. Switching to active-learning had no real effect. Gender made no difference. Intention of majoring in computer science was not a factor.
The other poster looked at measuring the style of the code in CS1 assignments automatically. They found that their tool was able to cluster the student submissions based on stylistic similarity and that grades for each cluster had a 90% confidence. I'm intrigued! Style is important and being able to emphasize style further is great.
And then I talked with other attendees for many hours, which is one of the reasons that I'm there.
Thursday, March 6, 2014
Conference Attendance SIGCSE 2014 - Afternoon Day 1
Concept inventory for Operating Systems! Develop an open concept inventory for a wider collection of classes, starting with an OS course. Work toward scenario questions to avoid issues with terminology. Consider page replacement, instead phrased about textbooks on a desk. Identify the concepts that are not intuitive by which answers have a higher percentage correct. They have a public share.
Process oriented guided inquiry learning in CS1. How pogil differs from active learning? Self managed teams with roles and they work through inquiry based activities, with the instructor as the facilitator. Maintain group composition over several weeks, while rotating roles. Split into two pairs for programming exercises. Both information retention (between CS1 and CS2) and female pass rates have improved. They noted a website carrying many developed resources.
Learning how to teach big data (at the middle school). A narrative game based environment to solve problems via pair programming. Worked first with middle school teachers to learn the CS concepts and then working with them to understand how to teach the students about big data. I'll need to read the paper to better follow this work, yet I am favorable toward pushing more CS content into earlier classes.
Assessment model for large project courses. How do you assess students on a large project when the students have different roles and focuses? Assessment variations: Formative vs summative. Teacher vs student. Group vs individual. Each project group is around 30 students. Grading criteria, oral feedback from instructor at student meetings, coaching from fellow students (code reviews, hackathons, etc), on demand artifacts, student reports (including contribution and time spent), individual teacher assessments (using a rubric), then the final feedback report for the group, and the option of interviews with individual students. And a final retrospective lead by the group. Most students are happy to have their grade based on the group performance. Few saw value of the reflective report.
A repository of novice programmer activity. Two million unique users using BlueJ every year. Blackbox collects anonymous data on the users. Data on each programming session, like compilation including result, line by line diffs of any edits. Then an interesting small analysis on the most common errors and how common the errors are over the duration of a course.
ACM exemplar course integrating fundamentals, programming languages, and software engineering. The problem is covering the increasing diversity of computing, yet reducing the credits required for a degree. This necessitates an integrated approach. The presentation followed with a description of the course, which showed the transitions between the integrated concepts. Several closed lab exercises exist to ready students for subsequent lectures.
Now it is time for the birds of a feather sessions. I am told that in the past there had been a session for students looking for a job, but there wasn't one this year. Still, I am intrigued by active learning in systems courses!
Process oriented guided inquiry learning in CS1. How pogil differs from active learning? Self managed teams with roles and they work through inquiry based activities, with the instructor as the facilitator. Maintain group composition over several weeks, while rotating roles. Split into two pairs for programming exercises. Both information retention (between CS1 and CS2) and female pass rates have improved. They noted a website carrying many developed resources.
Learning how to teach big data (at the middle school). A narrative game based environment to solve problems via pair programming. Worked first with middle school teachers to learn the CS concepts and then working with them to understand how to teach the students about big data. I'll need to read the paper to better follow this work, yet I am favorable toward pushing more CS content into earlier classes.
Assessment model for large project courses. How do you assess students on a large project when the students have different roles and focuses? Assessment variations: Formative vs summative. Teacher vs student. Group vs individual. Each project group is around 30 students. Grading criteria, oral feedback from instructor at student meetings, coaching from fellow students (code reviews, hackathons, etc), on demand artifacts, student reports (including contribution and time spent), individual teacher assessments (using a rubric), then the final feedback report for the group, and the option of interviews with individual students. And a final retrospective lead by the group. Most students are happy to have their grade based on the group performance. Few saw value of the reflective report.
A repository of novice programmer activity. Two million unique users using BlueJ every year. Blackbox collects anonymous data on the users. Data on each programming session, like compilation including result, line by line diffs of any edits. Then an interesting small analysis on the most common errors and how common the errors are over the duration of a course.
ACM exemplar course integrating fundamentals, programming languages, and software engineering. The problem is covering the increasing diversity of computing, yet reducing the credits required for a degree. This necessitates an integrated approach. The presentation followed with a description of the course, which showed the transitions between the integrated concepts. Several closed lab exercises exist to ready students for subsequent lectures.
Now it is time for the birds of a feather sessions. I am told that in the past there had been a session for students looking for a job, but there wasn't one this year. Still, I am intrigued by active learning in systems courses!
Conference Attendance SIGCSE 2014 - Morning Day 1
Here I am attending a Computer Science education conference. Started off with an interesting keynote, a break and meet a couple of faculty, and now the first paper session.
How to integrate software engineering into upper-level undergraduate courses? A project centered course, which included readings of selected research, as well as visiting local software development companies. Surveying the students before and after the class, and student confidence went down after the course, because students had learned how difficult the problems are. Yet students were more engaged into learning more about the subject and the resulting projects were of a higher quality.
Using real projects in software testing. Students, in teams, select a real world project. They develop a test plan, provide a progress report (requested by the students), and a final presentation. The instructor is both a customer and a coach. Able to work with the project developers. Target is generally low hundreds of classes in the project. Most students enjoyed the project and enrollment has increased. Students fill out a 360 survey on their teammates, and instructor intervention for outliers both positive and negative.
Student code is not throw aways. Best paper award. Prior work on software maintenance has usually relied on artificially prepared code, including lecturer added bugs. For this work, the code developed by prior seniors (11kloc, java, multithreaded) was provided to juniors in an intermediate version, who then added a feature and fixed bugs. Do students then follow proper practices? Most did, but a minority concluded, for example, that the code could not be tested. Many students observed the importance of quality design and had the experience of working on someone else's code base.
How to integrate software engineering into upper-level undergraduate courses? A project centered course, which included readings of selected research, as well as visiting local software development companies. Surveying the students before and after the class, and student confidence went down after the course, because students had learned how difficult the problems are. Yet students were more engaged into learning more about the subject and the resulting projects were of a higher quality.
Using real projects in software testing. Students, in teams, select a real world project. They develop a test plan, provide a progress report (requested by the students), and a final presentation. The instructor is both a customer and a coach. Able to work with the project developers. Target is generally low hundreds of classes in the project. Most students enjoyed the project and enrollment has increased. Students fill out a 360 survey on their teammates, and instructor intervention for outliers both positive and negative.
Student code is not throw aways. Best paper award. Prior work on software maintenance has usually relied on artificially prepared code, including lecturer added bugs. For this work, the code developed by prior seniors (11kloc, java, multithreaded) was provided to juniors in an intermediate version, who then added a feature and fixed bugs. Do students then follow proper practices? Most did, but a minority concluded, for example, that the code could not be tested. Many students observed the importance of quality design and had the experience of working on someone else's code base.
Friday, November 2, 2012
Computer Science Education: Student Conceptions
I'm sitting in Michael Hewner's thesis defense on "Student Conceptions About the Field of Computer Science". And I think there are some interesting points raised that are worth reflecting on. Let's start with summary questions.
Who would be considered a Computer Scientist?
How does a student choose his or her particular focus in Computer Science?
Do misconceptions about Computer Science affect the student's education?
Students have three conceptions about what Computer Science is, which were first derived from interviews and reaffirmed through a 100 student survey. The Theory-view (8%) is that Computer Science is mostly concerned with a theoretical view of computers, where the mathematical basis and understanding is dominant (although there may then exist a related field of Software Engineering). The Programming-view (41%) is that all Computer Science is about programs, where one is either analyzing the basis of or the direct work on programming. The Broad-view (27%) is that Computer Science is a giant umbrella of disciplines where computers are involved. A final 23% of the survey responses were without clear category, which may be due to the limitation of the original interviews. All conceptions view algorithms and data structures as a vital component to Computer Science.
Students use enjoyment of classes as the dominant metric of whether they have an affinity for the area. Ironically, one of the two dominant courses in my undergraduate education (15-213) was offered at 9am, which would commonly be viewed as an unpleasant time. Since students use enjoyment as their metric, students are not particularly affected by their misconceptions about what the course(s) contains.
Clearly then, the enjoyability of a course can then affect the taking of subsequent courses. Which in future work, there may be an exploration of whether course enjoyment effects (like scheduled time) has an effect on enrollment in follow-on courses in subsequent semesters. Furthermore, many students trust the curriculum as providing an adequate preparation for practicing Computer Science, which is to say that they are prepared as long as they satisfy the requirements regardless of any attempt to have a focus in their course selection. Should the curriculum then have unpleasant courses to force students into specializing?
For myself, I am a holder of the Programming-view (perhaps based on being a paid programmer), as I view Computer Science to be centered on programs and the act of programming. The field is directed toward understanding programs and how to program well. Computer Science is informed in part through Mathematics by providing a basis for understanding of algorithms and data structures, of which programs are fundamentally composed. Many fields, like Bio-Informatics, are related and rely on Computer Science, but are not Computer Science.
Wednesday, August 29, 2012
Computer Science Education (part 0 of N)
Besides my usual semesters of computer science courses and research, this fall I'm cross-enrolled at a neighboring university that offers education classes. Last night had some very interesting conversations. We are starting to prepare syllabi for the course we'd either ideally teach or want to be prepared to teach. I was in a group with two education PhDs (most of the class are). They consented to consider an introductory computer science course and answer two questions.
What are common naive theories that students have entering the course?
How might the course be designed to encourage students to reconstruct their theories?
So what naive theories do students have?
First, computers are magical. No, computers do exactly what a programmer tells them to do. (More advanced students learn about race conditions, compiler influence on correctness, etc). Which, unfortunately, means that if a computer is not doing what you want it to do, then you instructed it incorrectly (c.f., The rat is always right).
Second, I'm going to be a game programmer. No, most computer scientists do not write games (or at least, aren't paid to). But we find many other interesting parts to the field. Besides, many game programmers are treated little better than grad students.
Do you know other naive theories?
Then after class, I spent some time discussing more "advanced" theories in computer science.
Functional versus imperative programming. Does one paradigm exist to rule them all? Is one class of programming languages sufficient? Do students gain by learning about both paradigms? I discussed this briefly in Is versus ought, and have been regularly reading a strong function view in Existential Type.
Big 'O' notation and algorithm / data structure selection. I previously discussed this some in Know your N. And was co-author on a paper, "Brainy: effective selection of data structures", that demonstrated actual data structure selection for a program is not always best from the "Big 'O'" point of view.
Language equivalence. Related to functional versus imperative and one of my first posts, "Problem Solving via Programming", programming languages are theoretically equivalent (i.e., turning complete). But in practice languages should be selected for particular problems. What problems are best for specific languages?
What are some other major theories about computer science that students should know?
What are common naive theories that students have entering the course?
How might the course be designed to encourage students to reconstruct their theories?
So what naive theories do students have?
First, computers are magical. No, computers do exactly what a programmer tells them to do. (More advanced students learn about race conditions, compiler influence on correctness, etc). Which, unfortunately, means that if a computer is not doing what you want it to do, then you instructed it incorrectly (c.f., The rat is always right).
Second, I'm going to be a game programmer. No, most computer scientists do not write games (or at least, aren't paid to). But we find many other interesting parts to the field. Besides, many game programmers are treated little better than grad students.
Do you know other naive theories?
Then after class, I spent some time discussing more "advanced" theories in computer science.
Functional versus imperative programming. Does one paradigm exist to rule them all? Is one class of programming languages sufficient? Do students gain by learning about both paradigms? I discussed this briefly in Is versus ought, and have been regularly reading a strong function view in Existential Type.
Big 'O' notation and algorithm / data structure selection. I previously discussed this some in Know your N. And was co-author on a paper, "Brainy: effective selection of data structures", that demonstrated actual data structure selection for a program is not always best from the "Big 'O'" point of view.
Language equivalence. Related to functional versus imperative and one of my first posts, "Problem Solving via Programming", programming languages are theoretically equivalent (i.e., turning complete). But in practice languages should be selected for particular problems. What problems are best for specific languages?
What are some other major theories about computer science that students should know?
Thursday, May 10, 2012
The Minimum of Computer Science
The NY Times ran an article at the start of last month on what CS education is required for non-majors, Computer Science for Non-Majors. Is the knowledge of how to write a (simple) program so vital to modern work that it should rank with multiplication tables, English grammar, etc? And if so, how much knowledge and in what language(s) should this be taught?
Should someone learn the basics of javascript, so they can add to webpages? Perhaps a little python to do simple processing of data? But probably not C, as the language expresses what the computer is to do and not what the computer should do.
Edit: A strong response to this general question here.
Should someone learn the basics of javascript, so they can add to webpages? Perhaps a little python to do simple processing of data? But probably not C, as the language expresses what the computer is to do and not what the computer should do.
Edit: A strong response to this general question here.
Subscribe to:
Posts (Atom)