Repost: Code Smells ... Is concurrency natural?

Writing parallel code is not considered easy, but it can be a natural approach to some problems for novices.  When a beginner wants something to happen twice concurrently, the reasonable thing would be to do what works once, a second time.  Instead, this may conflict with other constructs of the language, such as main() or having to create threads.  See more here.

Conference Attendance - SIGCSE 2019 - Day 2.5

Continuing at SIGCSE, here are several more paper talks that I attended on Friday.  Most of the value at SIGCSE comes from the friendly conversations with other attendees.  From 5-11p, I was in the hotel lobby talking with faculty and students.  Discussing research ideas, telling our stories from teaching, and generally renewing friendships within the field.

On the Effect of Question Ordering on Performance and Confidence in Computer Science Examinations
On the exams, students were offered a bonus if they could predict their score by within 10%.  Does the order of questions (easy -> hard, or hard -> easy) have any impact on their estimated or actual performance on an exam.  Students overpredicted by over 10% on the exams.  As a whole, the hard to easy students did worse, but this result was not statistically significant.  A small improvement is gained for women when the exams start with the hardest problem.

I wonder about whether students were biased in their prediction based on the reward.  Ultimately, the authors gave the reward to all students regardless of the quality of their prediction.

The Relationship between Prerequisite Proficiency and Student Performance in an Upper-Division Computing Course

We have prerequisites to ensure that students are prepared for the later course, an upper-level data structures class.  Students started on average with 57% of expected prerequisite knowledge, and will finish the course with an improvement of 8% on this knowledge.  There is a correlation between prerequisite score and their final score.  With several prerequisites, some knowledge concepts has greater correlation than others.  Assembly is a surprising example of a concept that relates.  Students benefit from intervention that addresses these deficiencies early in the term.

Afterward, we discussed that this work did not explore what prerequisite knowledge weakly correlated with student learning.  How might we better understand what prerequisites actually support the learning in a course?  Furthermore, can we better understand the general background of students in the course, such as class standing or general experience?

Visualizing Classic Synchronization Problems

For three classic synchronization problems: dining philosophers, bounded producer-consumer, and readers and writers.  Each one has a window displaying the operations, as well as multiple algorithmic strategies.  With these visualizations, do students learn better and also find them more engaging than reading about the problems in the textbook.  While not statistically significant, the control group exhibited better recall, although the visualization group had higher engagement.  That said, the control group exhibited higher course grades, so the difference in learning may actually be from unrelated factors.

Conference Attendance: SIGCSE 2019 - Day 1.5

Back at SIGCSE again, this one the 50th to be held.  Much of my time is spent dashing about and renewing friendships.  That said, I made it to several sessions.  I've included at least one author and linked to their paper.

Starting on day 2, we begin with the Keynote from Mark Guzdial

"The study of computers and all the phenomena associated with them." (Perlis, Newell, and Simon, 1967).  The early uses of Computer Science were proposing its inclusion in education to support all of education (1960s).  For example, given the equation "x = x0 + v*t + 1/2 a * t^2", we can also teach it as a algorithm / program.  The program then shows the causal relation of the components.  Benefiting the learning of other fields by integrating computer science.

Do we have computing for all?  Most high school students have no access, nor do they even take the classes when they do.

Computing is a 21st century literacy.  What is the core literacy that everyone needs?  C.f. K-8 Learning Trajectories Derived from Research Literature: Sequence, Repetition, Conditionals.  Our goal is not teaching Computer Science, but rather supporting learning.

For example, let's learn about acoustics.  Mark explains the straight physics.  Then he brings up a program (in a block-based language) that can display the sound reaching the microphone.  So the learning came from the program, demonstration, and prediction.  Not from writing and understanding the code itself.  Taking data and helping build narratives.

We need to build more, try more, and innovate.  To meet our mission, "to provide a global forum for educators to discuss research and practice related to the learning and teaching of computing at all levels."

Now for the papers from day 1:

Lisa Yan - The PyramidSnapshot Challenge

The core problem is that we only view student work by the completed snapshots.  Extended Eclipse with a plugin to record every compilation, giving 130,000 snapshots from 2600 students.  Into those snapshots, they needed to develop an automated approach to classifying the intermediate snapshots.  Tried autograders and abstract syntax trees, but those could not capture the full space.  But!  The output is an image, so why not try using image classification.  Of the 138531 snapshots, they generated 27220 images.  Lisa then manually labeled 12000 of those images, into 16 labels that are effectively four milestones in development.  Then, a neural network classifier classified the images.  Plot the milestones using a spectrum of colors (blue being start, red being perfect).  Good students quickly reach the complete milestones.  Struggling students are often in early debugging stages.  Tinkering students (~73 percentile on exams) take a lot of time, but mostly spend it on later milestones.  From these, we can review assignments and whether students are in the declared milestones, or if other assignment structure is required.

For the following three papers, I served as the session chair.

Tyler Greer - On the Effects of Active Learning Environments in Computing Education

Replication study on the impact of using an active learning classroom versus traditional room.  Using the same instructor to teach the same course, but using different classrooms and lecture styles (traditional versus peer instruction).  The most significant factor was the use of active learning versus traditional, with no clear impact from the type of room used.

Yayjin Ham, Brandon Myers - Supporting Guided Inquiry with Cooperative Learning in Computer Organization

Taking a computer organization course with peer instruction and guided inquiry, can the peer instruction be traded for cooperative learning to emphasize further engagement and learning.  Exploration of a model (program, documentation), then concept invention (building an understanding), then application (apply the learned concepts to a new problem).  Reflect on the learning at the end of each "lecture".  In back-to-back semesters, measure the learning gains from this intervention, as well as survey on other secondary items (such as, engagement and peer support).  However, the students in the intervention group did worse, most of which is controlled by the prior GPA.  And across the other survey points, students in the intervention group rated lower.  The materials used are available online.

Aman, et al - POGIL in Computer Science: Faculty Motivation and Challenges

Faculty try implementing POGIL in the classroom.  Start with training, then implementing in the classroom, and continued innovation.  Faculty want to see more motivation, retaining the material, and staying in the course (as well as in the program).  Students have a mismatch between their learning and their perceived learning.  There are many challenges and concerns from faculty about the costs of adoption.

Seminar Talk: Computer Science Pedagogy - Miranda Parker

This week I have been co-hosting Miranda Parker, from Georgia Tech, as part of our Colloquium on Computer Science Pedagogy.  Her work is titled, Barriers to Computing: What Prevents CS for All.

The question is what are the barriers to accessing Computer Science at the high school level.  Nation-wide, ~35% of high schools offer at least one CS course, although this is self-reported CS.  In Indiana, the most popular courses are taken by ~5000 students, state-wide.  In Austin Texas, about 6000 students take at least one CS course, out of ~110,000.

How do we know if students are successful (i.e., did they learn)?

For this, we need a validated assessment to ensure that we are measuring what we want to measure.  An initial assessment, FCS1, worked fairly well and across multiple introductory programming languages; however, the assessment had limits in its use, which lead to the SCS1.  This assessment correlated well with FCS1, so it can standin; however, it was an hour long.  Assessments should: cover the content, vary in difficulty, and have good discrimination (so individual scores are good predictors for the overall performance).  In analysis, most of the SCS1 questions were hard, and few provided good discrimination.  The assessment was then adapted to focus on the medium difficulty problems that discriminate (in test scores), and expanded to a 30 minute test.

With a measurement of whether students are succeeding in Computer Science, we can walk back to investigate what things influence students to succeed in Computer Science.

Among prior studying factors, we know that students do better in CS if they have more prior experience with Computer Science, and students do better in CS (as well as STEM and general education) with higher socioeconomic status (SES).  There are also known prior links between SES and access to computing (whether it is formal courses, or informally), and SES to spatial reasoning.  And both of these later components are linked to CS achievement.

In exploratory study (large, public university with mainly high SES students), showed statistical correlation from spatial reasoning to CS achievement.  There was not correlation from having access to achievement.  In the interest of time, this point was not presented further.

Barriers to computing

There are three main sources of access to Computer Science courses: state policies, geography (such as, circumstances, industry, or neighboring schools with CS), and resources (such as, time and money or rural versus urban).

In partnership with Georgia Department of Education, CS enrollment data from 2012-2016, plus other public data (such as, characteristics of the county and of the school), for each of the 181 school districts in Georgia, where each district has at least one high school.  Using the definition of CS courses, being those that count toward the graduation requirement in Georgia as computer science, versus other computing courses, such as web design or proficiency with office.  In Georgia, out of 500,000 students, about 6000 took a CS course in a given year, where about 50% of schools offered computer science during at least one year in that time frame.  And the 6000 students are actually student course events, where a student taking two CS courses would count twice.

For those schools, CS enrollment, total high school enrollment, and median income contribute to whether CS will be offered in the next year.

This work is yet ongoing, where the next steps are to visit schools and collect further data on these factors, such as why a school discontinued a course offering, or now offers one.  Or what do students these courses go on to do?  An audience question wondered whether the CS courses offered relates to the courses offered of other parts of STEM.

Conference Attendance SIGCSE 2018

I have just finished attending SIGCSE 2018 in Baltimore.  In contrast to my earlier conference attendance, this time I have had higher involvement in its execution.

On Wednesday I went to the New Educator's Workshop (NEW).  Even being faculty for two years, there was still a number of things that were either new or good reminders.  Such as including or discussing learning objectives with each lecture and assignment, or being careful with increasing one's level of service.  As a new faculty member, each service request seems exciting, as no one has asked me before!  But many senior faculty emphasized that this is the time in which they are protecting us from lots of service opportunities such that we can spend time on our teaching and research.

On Thursday morning, I presented my recent work that updated a programming assignment in Introduction to Computer Systems, and from which we saw improvements in student exam scores.  We did not research the specific action, and are therefore left with two theories.  First, the improvement could be from using better style in the starter code and emphasizing this style in submissions.  Second, we redesigned the traces to require submissions to address different cases and thereby implement different features.  I lean toward the formed, but have no data driven basis for this hypothesis.

Let's discuss active learning briefly.  I attended (or ran) several sessions focused on this class of techniques.  The basic idea is that students have better engagement and learning by actively participating in class.  There are a variety of techniques that work to help increase student activity.  On Thursday afternoon, Sat Garcia of USD, presented Improving Classroom Preparedness Using Guided Practice, which showed how student learning improved from participating in Peer Instruction, which particularly requires students to come to class prepared.  Shortly later, Cynthia Taylor joined Sat and I in organizing a Bird of Feather (BoF) session on using Active-learning in Systems Courses.  We had about 30-40 attendees there split into two groups discussing some techniques they have used and problems they have observed.  5 years ago, a similar BoF had attendance around 15-20, so we are making progress as a field.

On Friday, I spoke with Brandon Myers who has done work on using POGIL in Computer Organization and Architecture.  In POGIL, students are working in groups of 3-4 with specific roles through a guided learning, guiding students into discovering the concepts themselves.  We had a nice conversation and may be merging our draft resources.  This last point is often the tricky part of using active learning in that developing reasonable materials can be both time intensive and requires several iterations.

The Friday morning keynote presentation was given by Tim Bell, who spoke about K-12.  This topic is rather distant from my own work and research, so I was skeptical.  Yet, I came out quite enthused.  It was interesting to think about presenting Computer Science concepts in non-traditional ways, based initially on having to explain your field at elementary school when the other presenters are a cop and a nurse (his example).  How could you get 6 year olds to sort?  Or see the advantage of binary search as the data grows?

In the afternoon, I was a session chair for the first time.  I moderated the session on Errors, so obviously the AV system stopped working for a short duration.  Beyond that incident, the session seemed to go well.

I always like going to SIGCSE.  It is rejuvenating and exhausting.  So many teachers to speak with about courses, curriculum, and other related topics.  And then you find that you've been social for 16 hours or so hours.

The Importance of Debugging

How do you teach students about debugging?  To have The Debugging Mind-Set?  Can they reason about possible causes of incorrect behavior?

For the past year, I have been revising material to help students learn about using gdb to assist in debugging, which is an improvement over the "printf-based" methods previously.  And while this approach is usually used when the program has crashed from a segfault, many students are stymied when the problem is incorrect behavior rather than invalid behavior.

When their program crashes, they usually appreciate that gdb can show them what line of code / assembly has crashed.  But how can a student "debug" incorrect behavior?  Many try the "instructor" debugging method (they try this too when the code is crashing), where they either present their code or describe the basics of what they have done and ask us, as an oracle, what is wrong.  I try to offer questions that they need to answer about their code.  Sometimes the student follows well and this is valuable guidance for him or her to solve the behavior issue.

Other times I have asked these questions, trying to build up a set of hypotheses to test and investigate, and the student effectively rejects them.  Not for being wrong, but for not clearly being the answer.  They have the subconscious idea that their code is failing for reason X, which was their intuitive guess (these guesses are a good start).  But the idea that they just do not know enough and need to collect more data is not grasped.

You are a doctor when debugging.  Sometimes the patient gives clear symptoms.  And other times, you need to run more tests.  Again, thankfully, usually if an instructor recommends running a certain test, the data gleaned is enough to guide them through the diagnosis.  Students appreciate when this happens; however, there is creativity in considering other possibilities and sometimes that possibility requires being open to everything (see TNG Finale).

This semester I am co-teaching Operating Systems.  We have told the students that you have to know how to debug, as sometimes printf is what you have to debug.  And other times, to quote James Mickens, "I HAVE NO TOOLS BECAUSE I’VE DESTROYED MY TOOLS WITH MY TOOLS."  So in the dystopian, real-world, you need all the debugging tools and techniques you can get.

PhD Defense - Automated Data-Driven Hint Generation for Learning Programming

Kelly Rivers defended her PhD work this afternoon.  She will returning to CMU this fall as a teaching professor.

Student enrollment is increasing, so more work is needed to automate the support, as TAs / instructors are not scaling.  Prior work (The Hint Factory) developed models based on prior student submissions, and then a current student's work can be found within the model thus providing suggestions for how to proceed.  However, programming may not fit within this model due to the larger and more varied space for which students can solve the problems.

First, student code proceeds through a series of canonicalization steps - AST, anonymized, simplification.  Such that the following python code is transformed:

import string
def any_lowercase(s):
  lst = [string.ascii_lowercase]
  for elem in s:
    if (elem in lst) == True:
      return True
    return False

Becomes

import string
def any_lowercase(p0):
  for v1 in p0:
    return (v1 in string.ascii_lowercase)

Studies then went over 41 different problems with hundreds of correct solutions and thousands of incorrect solutions.  The model can then generate the edits and chain these hints as necessary.  In more than 99.9% of cases, the model could successfully generate a hint chain to reach a correct solution.

To further test this model and approach, the model started with the empty space (just teacher solution) and was compared against the final model.  Ideally, the final model will propose fewer edits than the initial model.  And for 56% of problems, this was true.  40% of problems were already optimal.  And 3% are opportunities for improvement to the model.

Next, given this model exists, how do the hints impact student learning?  Select half of the students to give them access to the hint model optionally.  Using a pre / post assessment, the measurement was a wash.  Instead, a second study was designed that required the students to use the system within a two hour OLI module.  Hints would be provided with every submission and either before or after the midtest in the OLI module.  Only 1/2 of the students actually proceeded through the module in order.  However, most learning was just within the pretest->practice->midtest, so adding those students increased the population.  The results show that the hints reduce the time required to learn the equal amount.

From interviews with students, students need and want targeted help on their work.  However, the hints generated thus far were not always useful.  Proposed another study based on different styles of hints: location, next-step, structure, and solution.  This study found that participants with lower expertise wanted more detailed hints.  Hint usage would sometimes be for what is wrong versus how to solve it.  And often, students know what to do, and just need to reference (via example / prior work) how to do this, rather than hinting what to do.

Computer Scientists and Computers Usage

This post is built on a discussion I had with Dr. Thomas Benson and the recent blog post by Professor Janet Davis, I've got a POSSE.  Less directly, Professor Mark Guzdial has written several recent blog posts about parents wanting CS in high school, most recently this (where the point is raised that not everyone knows what this is).

What is it to be a computer scientist?  Is it just software development?  When I was first looking at colleges and I knew I wanted to program, I saw three majors that seemed appropriate: Computer Science, computer programming, and game development.  And from my uninformed perspective, Computer Science (CS) initially seemed the least applicable.  This was before Wikipedia, so how would one know what CS is?

Michael Hewner's PhD defense was a study of how undergrads perceive the field, as they learn and progress through the curriculum, do their views change?  I know my views changed; for example, functional programming was a complete unknown before matriculating.  By the completion of my undergraduate degree, I perceived that Computer Science has three pillars: systems, theory, and application.  I still view CS primarily from a programmer's lens and not a big tent view.

Indirectly, the Economist wrote about Programming Boot Camps where college grads go back to get a training in programming; however, "Critics also argue that no crash course can compare with a computer-science degree. They contend that three months’ study of algorithms and data structures is barely enough to get an entry-level job."  I both agree and disagree.  There is a further transformation of the economy coming whereby workers will .  One PL (programming language) researcher recently estimated that Excel is the most common programming language.  Can the worker using Excel do more or do it faster with VB or python or ...?

Are there nuances to the study of Computer Science in which students can focus further?  Clearly there are with the mere presence of electives.  Georgia Tech even groups similar electives into "threads".  Besides just electives, under a programmer-centric view of CS, software development involves more than just the strict programming aspect.  Even if CS is the "programming", a written program requires software engineering to maintain and manage its development and designers to prepare the UI (etc).

Thus CS might evolve more toward having a pre-CS major, and after the first two(?) years, students can then declare as Computer Science or Software Engineering or Human Computer Interaction or ....  So this is both less and more than Georgia Tech's threads, but an approach with similarities.  In common, this development of CS major(s) balances a concern of whether students have the fundamentals to succeed in areas related (i.e., programming, software development, and design) to their actual major, while allowing some greater depth and specialization.

This step is a maturing of the discipline.  No long just one major, but a school of majors.  CMU offers BS in CS, along with three other interdisciplinary majors.  Georgia Tech offers CS (with 8-choose-2 threads) and Computational Media.  Rose-Hulman offers both CS and Software Engineering.  (I am only citing the programs that I know about, not an exhaustive list).

Conference Attendance SIGCSE 2016 - Day 2

After lunch when we are all in food comas, let's attend the best paper talk!
A Multi-institutional Study of Peer Instruction in Introductory Computing -
This study followed 7 instructors across different institutions as they used peer instruction.  This showed that both the instruction is generally recognized as valuable, while also touching on routes in which it can go awry.  Tell students why this technique is being used and what it's effect.  Hard questions are good questions to ask, as students will discuss and learn from the question.  This requires that questions are graded for participation and not *correctness*.  Possible questions and material for peer instruction is available.

Development of a Concept Inventory for Computer Science Introductory Programming -
A concept inventory is a set of questions that carefully tease out student misunderstandings and misconceptions.  Take the exams and identify both the learning objective and the misconception that results in incorrect answers.

int addFiveToNumber(int n)
{
  int c = 0;
  // Insert line here
  return c;
}

int main(int argc, char** argv)
{
  int x = 0;
  x = addFiveToNumber(x);
  printf("%d\n", x);
  return 0;
}

a) scanf("%d", &n);
b) n = n + 5;
c) c = n + 5;
d) x = x + 5;

Each incorrect answer illustrates a different misconception.  For example, input must come from the keyboard.  Or variables are passed by reference.
Overall, this study illustrated how the concept inventory was developed, but not the impact of having it, or what it showed in the students and their learning.

Uncommon Teaching Languages - (specifically in intro courses)
An interesting effect of using an uncommon language in an introductory course is that the novices and experts have similar skills.  Languages should be chosen to minimize churn, otherwise students feel that they haven't mastered any languages.  And related to this point, languages also exist in an institutional ecosystem.  Furthermore, we want to minimize the keywords / concepts required for a simple program.  A novice will adopt these keywords, but they also are "magic" and arcane.  And then how long are the programs, as we want novices to only have to write short code to start.

I also attended the SIGCSE business meeting and then the NCWIT reception.  I have gone to NCWIT every year at SIGCSE, as I want to know what I should do (or not do) to not bias anyone's experience in Computer Science.

Conference Attendance SIGCSE 2016 - Day 1

Here I am at SIGCSE again.  This is a wonderful opportunity to think and reflect on how I assist students in learning Computer Science and to be Computer Scientists.  And to connect with other faculty, researchers, etc who are interested in teaching and doing so in a quality manner.

An Examination of Layers of Quizzing in Two Computer Systems Courses -
In this work, the instructor taught the Intro Computer Systems course and based on Bryant and O'Hallaron's book (paid link).  After several years of teaching, she introduced a new layer of quizzing to the course.  Effectively before each class, students take a pre-quiz worth ~0% of their grade (20 quizzes combine to 5%), and can then come to class with knowledge and feedback toward their deficiencies.  From the experience of the quizzes, students have been doing better in these courses.

Subgoals Help Students Solve Parsons Problems -  (previewed at Mark Guzdail's blog)
When learning new things, students benefit from labeling subgoals in solving.  These labels provide a basis for solving similar problems.  There are two different strategies for labeling: students can provide the labels or the assignment can provide the labels.  An example labeling can be found with loops: initialize, test, change.  If students provide the labels and provide cross-problem labels, they do best.  If they provide the labels and they are problem-specific such as "are there more tips" (with respect to an array of tips), then these students do worse than those provided the labels.  Developing labels can be valuable, but it may require the expert to still provide guidance to help abstract them across problems.  This talk had one of the great moments when someone asked a question and Brianna replied by, "So and so has done great ..."  And the questioner pointed out that he is "so and so".

As CS Enrollments Grow, Are We Attracting Weaker Students?: A Statistical Analysis of Student Performance in Introductory Programming Courses Over Time -
In this study, one instructor has analyzed the data of student assignment grades across 7 years of Fall semesters in the CS 1 course.  Several specific and clear reasonings were applied to get a clear and comparable data set.  The first test is that the number of student withdrawals remained the same as a percentage of the total class size.  The second test is that the means of the grades for the courses are statistically indistinguishable.  The third test is to use a mixture model (weighted combination of distributions) for each class's scores.  A good fit is found with two gaussian distributions, such that there is one for the "good students" and a second for the high variance students who are "potentially weaker".  From this, the study concluded that (at Stanford, in Fall CS1), there are more "weak students" and more "strong students" as the student enrollment is drawing from the same larger population.

A (Updated) Review of Empiricism at the SIGCSE Technical Symposium -
Using the proceedings from SIGCSE 14 and 15, they examined the empirical evaluation and the characteristics of these evaluations.  How was the data collected in each paper?  And what was being evaluated (pedagogy, assignments, tools, etc)?  Is the subject novel or replicating other studies?  Based on this study, would SIGCSE benefit from a separate track for longer paper submissions?  Or workshops on how to empirically validate results?  This and other material is being developed under an NSF grant and released publically.

Birds of a Feather -
In the evening, I attended two Birds of a Feather sessions.  Both of which have given me further ideas for what I might do to further (attempt to) improve student learning.  And also possible collaborators toward that end.

Teaching Inclusively in Computer Science

When I teach, I want everyone to succeed and master the material, and I think that everyone in the course can.  I only have so much time to work with and guide the students through the material, so how should I spend this time?  What can I do to maximize student mastery?  Are there seemingly neutral actions that might impact some students more than others?  For example, before class this fall, I would chat with the students who were there early, sometimes about computer games.  Does those conversations create an impression that "successful programmers play computer games"?  To these questions, I want to revisit a pair of posts from the past year about better including the students.

The first is a Communications of the ACM post from the beginning of this year.  It listed several seemingly neutral decisions that can bias against certain groups.  Maintain a tone of voice that suggests every question is valuable and not "I've already explained that so why don't you get it".  As long as they are doing their part in trying to learn, then the failure is on me the communicator.

The second is a Mark Guzdial post on Active Learning.  The proposition is that using traditional lecture-style advantages the privileged students.  And a key thing to remember is that most of us are the privileged, so even though I and others have "succeeded" in that setting, it may have been despite the system and not because of the teaching.  Regardless of the instructor, the teaching techniques themselves have biases to different groups.  So if we want students to master the material, then perhaps we should teach differently.

Active learning has a growing body of research that shows using these teaching techniques help more students to succeed at mastering a course, especially the less privileged students.  Perhaps slightly less material is "covered", but students will learn and retain far more.  Isn't that better?

Repost: Incentivizing Active Learning in the Computer Science Classroom

Studies have shown that using active learning techniques improve student learning and engagement.  Anecdotally, students have brought up these points to me from my use of such techniques.  I even published at SIGCSE a study on using active learning, between undergraduate and graduate students.  This study brought up an interesting point, that I will return to shortly, that undergraduate students prefer these techniques more than graduate students.

Mark Guzdial, far more senior than me, recently challenged Georgia Tech (where we both are) to incentivize the adoption of active learning.  One of his recent blog posts lists the pushback he received, Active Learning in Computer Science.  Personally, as someone who cares about the quality of my teaching, I support these efforts although I do not get to vote.

Faculty members at R1 institutions, such as Georgia Tech, primarily spend their time with research; however, they are not research scientists and therefore they are being called upon to teach.  And so you would expect that they would do this well.  In meeting with faculty candidates, there was one who expressed that the candidate's mission as a faculty member would be to create new superstar researchers.  Classes were irrelevant to this candidate as a student, therefore there would be no need to teach well as this highest end (telos) of research justifies the sole focus on students who succeed despite their instruction, just like the candidate did.  Mark's blog post suggests that one day Georgia Tech or other institutions may be sued for this sub-par teaching.

What about engagement?  I (along with many students and faculty) attended a visiting speaker talk earlier this week and was able to pay attention to the hour long talk even though it was effectively a lecture.  And for this audience, it was a good talk.  The audience then has the meta-takeaway that lectures can be engaging, after all we paid attention.  But we are experts in this subject!  Furthermore, for most of us there, this is our subfield of Computer Science.  Of course we find it interesting, we have repeatedly chosen to study it.

For us, the material we teach has become self-evidently interesting.  I return to the undergraduate and graduate students that I taught.  Which group is closer to being experts?  Who has more experience learning despite the teaching?  Who prefered me to just lecture?  And in the end, both groups learned the material better.

Edit: I am by no means condemning all of the teaching at R1's or even Georgia Tech.  There are many who teach and work on teaching well.  The Dean of the College of Computing has also put some emphasis on this through teaching evaluations.  Mark's post was partially noting that teaching evaluations are not enough, we can and should do more.

Course Design Series (Post 2 of N): Choosing a Textbook

Having now read both Programming Language Pragmatics, Third Edition and Concepts of Programming Languages (11th Edition), I have settled on the former as my textbook for the fall.  I do not find either book ideally suited, and I wish that the Fourth Edition was being released this summer and not in November, which is why it still hasn't arrived.

For the choice of which textbook to use, it was "Concepts" to lose and having 11 editions, the text should also be better revised.  I dislike reading the examples in a book and questioning how the code would compile.  Beyond which, the examples felt quaint and contrived.

(edit) For example, I was reviewing the material on F# and copied in an example from the text:
let rec factorial x =
  if x <= 1 then 1
  else n * factorial(n-1)
Does anyone else notice that the function parameter is x on the first two lines and n on the last?!

Before the Concepts book is written off entirely, there are many valuable aspects.  I enjoyed reading about the history of programming languages, especially for exposing me to Plankalkül.  The work also took a valuable track in the subject by regularly looking at the trade-offs between different designs and features.  This point certainly helped inform my teaching of the material.

Fundamentally, when I looked at the price of the two textbooks, the benefits of using the newer Concepts textbook could not outweigh the nearly doubled pricetag.  Most of the positive points are small things and can be covered as addendums to the material.

(FCC note - Concepts of Programming Languages was provided free to me by the publisher.)

Conference Attendance FCRC - Day 1 - WCAE / SPAA

In Portland for the next 5 days attending the Federated Computing Research Conference, which is a vast co-location of the top ACM conferences.  For my part, this includes ISCA and PLDI.  Following registration and checking in as a student volunteer, I ducked in to the Workshop on Computer Architecture Education (WCAE).  There were a couple of presentations on different tools being used to teach architectural concepts.

Following the morning break, it was time for the keynote for SPAA, given by Hans-J Boehm, titled, "Myths and Misconceptions about Threads".  For example,

#include "foo"
f() {
    foo_t x, a;
    ...
    x = a; // Is this atomic?
}

Which lead to the discussion of 'is assignment atomic?' and the audience tossed out increasing complex examples of how it is not.  Fundamentally, the programming model is becoming "data-race free", and the specifications can treat races as "undefined behavior".  In general, a sequential program will view its execution following the sequential consistency model, even if the hardware is executing the code with a weaker model.

What then should the programming language provide for the atomics / synchronization?  Recall that the compiler has considerable flexibility for emitting the final program.  With data-race free code, the compiler is treating anything that is not an atomic as part of sequential code and therefore subject to any reordering that would still be valid sequentially.  The following example is how this can go awry.  X is a global, and the compiler could substitute x anyplace tmp is, because the model assumes "there are no races on x".  And if the program does happen to modify x is a racy manner, then the behavior is undefined.

bool tmp = x;
if (tmp) f = new ...
...
if (tmp) f->foo();

Gah!  The programmer wanted to take a snapshot of the global value, but ended up with a different result.  So the atomics are becoming more than just the "hacker's" way to quickly update shared values, and instead can be seen as annotations to the compiler to clearly encapsulate the shared state.  This means the type of x is not bool, but atomic<bool>.  Then the compiler knows the programmer's (likely) intent of this code.  And this then rolls back to a deeper question of my research, "What could the system do more efficiently if it knew more about the programmer's intent?"

Course Design Series (Post 0 of N): A New Course

This fall I will again be Instructor of Record.  My appointment is to teach CS 4392 - Programming Languages.  This is an unusual situation in that the course has not been taught for over 5 years (last time was Fall 2009).  Effectively, the course will have to be designed afresh.

The first step in course design (following L. Dee Fink and McKeachie's Teaching Tips: Chapter 2) is to write the learning objectives using the course description, along with prerequisites and courses that require this one.  Let's review what we have:

Course Description: none

Prerequisites:
Undergraduate Semester level CS 2340 (which has the following description)
Object-oriented programming methods for dealing with large programs. Focus on quality processes, effective debugging techniques, and testing to assure a quality product.

Courses depending on this: none

Alright.  Now I will turn to the CS curriculum at Georgia Tech.  Georgia Tech uses a concept they call "threads", which are sets of related courses.  CS 4392 is specifically in the Systems and Architecture thread.  This provides several related courses:

CS 4240 - Compilers, Interpreters, and Program Analyzers
Study of techniques for the design and implementation of compilers, interpreters, and program analyzers, with consideration of the particular characteristics of widely used programming languages.

CS 6241 - Compiler Design
Design and implementation of modern compilers, focusing upon optimization and code generation.

CS 6390 -  Programming Languages
Design, structure, and goals of programming languages. Object-oriented, logic, functional, and traditional languages. Semantic models. Parallel programming languages.

Finally, the ACM has provided guidelines for the CS curriculum.  Not only does this provide possible options for what material I should include, but they have also provided several ACM exemplar courses (c.f., http://www.cs.rochester.edu/courses/254/fall2013/ and http://courses.cs.washington.edu/courses/cse341/13sp/).

To summarize, if my first step is to write the learning objectives, then I am on step 0: write the course description.  In a couple of weeks, I plan on finishing my initial review of potential textbooks as well as the other materials covered above.  That will provide me the groundwork for the description and then objectives.

Conference Attendance SIGCSE 2015 - Day 2 / 3

I recognize that Day 1 afternoon went “missing”.  I presented my poster and that consumed the sum total of my time.  While I am happy with all that I achieved with my poster (writing IRB protocol, independent work, analyzing my teaching, et cetera), it was not considered as a finalist for the student research competition (SRC). Yet I received significant feedback and a number of follow-ons that I will have to try to evaluate the next time(s) I teach.  I have been doing an excellent job of networking and speaking with my colleagues.  And I have seen several exciting techniques to improve my teaching.

BOF2: Preparing Undergraduates to Make the Most of Attending CS Conferences

This was a small Bird of the Feather (BoF), but we discussed some approaches and things to know before attending your first conference, and not just for undergraduates.  However, in some cases, undergraduates are attending their local institution and may never have traveled any significant distance.  What do they really need to bring?



In traveling, take some time to prepare students.  Let them know what to expect.  For example, it is okay to miss some paper sessions, and even return to your room entirely.  It is okay to ask questions 1:1.  Find groups where people are being introduced and join in.  Student volunteering, while takes time, also gives an additional individuals that you will know.  Use the people you know to introduce you to others at the conference.

MIPSUnit: A Unit Testing Framework for MIPS Assembly

This is just what it sounds.  A Ruby based framework that enables writing simple unit tests that will then be applied to a full simulation of the assembly executed.

A Web-based Simulator for Learning Multiprocessor Real-time Scheduling

The presenter(s) were not at this poster, but it showed a high quality interface for seeing the scheduling of threads according to different scheduling policies.  The intent here was not to explore races and parallelism, but rather see how scheduling decisions are made in an OS.

Raspberry HadooPI: A Low-Cost, Hands-On Laboratory in Big Data and Analytics

I was not expecting this poster.  You are walking along and then see 4 Raspberry Pi's all networked together.  Raspberry Pis and HPC?!  A small setup, but it is an interesting development that takes advantage of the low cost Pi and still provide an HPC platform for students.

Building the Pascaline: Digital Computing Like It's 1642 

Plastic parts all worked together to form replicas of Pascal's mechanical calculator.  Interesting and student assembled.

On the Evaluation of Student Team Software Development Projects

Teams of 4 students, approach is evaluated on courses from three years of major (sophomore on up).  Teams are formed with CATME (particularly using dissimilar GPAs in a group), as well as partner selection (when possible).  Students provide peer evaluations after each stage of the project.  Significant data collection looking particularly at what students prefer for to be the evaluation policy (between 100% of grade for the group’s work to 100% of the grade for the individual’s contribution).  This question was taken repeatedly throughout the semester, which leads to whether student preferences change?  More senior students prefer more weight being attributed to group.  The predictor for what grade split is at what point in the course is this surveyed, and effectively as soon as the teams are formed the students prefer to be graded primarily as a group.  Follow on study is looking at experience with team projects, trust in the ability to evaluate individual contribution, and other questions.  This is a hopeful data point.

What Influences CS Faculty to Adopt Teaching Practices?

How do faculty become aware and why do they try out teaching practices?  66 participants in CS, including chairs, tenure-track faculty, teaching faculty, and Ph.D. student instructors across 36 institutions.  First, the mental model of what an instructor does can differ significantly from what the instructor is actually doing.  Second, faculty can find out about practices through a variety of approaches, such as self-identifying that there is possible improvement in their teaching.  Faculty often trust other faculty like them (researchers to researches, lecturers to lecturers).  Third, when adopting a practice, faculty need to evaluate the effectiveness (see also my poster, student feedback, etc).  -- My efforts in this have been having different faculty (my recommendation letter writers) view my lectures / teaching, and thereby giving them demonstrations of different practices.

Personalized Attention @ Scale

"We lost the war on cheating"  Instead, we have to meet with students such that they are demonstrating their understanding of the code.  The requirements of submissions: attribute your sources and understand your submission.  Enables students to work together, use all sources, develop interview skills.  Enables reuse of assignments.  Grading is now 40% correctness / 60% code interview.  Rubric for each interview.  Students should arrive early and have their laptop ready to present / explain.  Students were better able to learn and complete the assignments, as well as feedback for improvement.  Students also felt better able to learn the material by being able to collaborate and not constrained by a collaboration policy.  There are some stressors, such as TAs having to meet with hundreds of students, as well as their inconsistencies.  -- This was perhaps the most exciting new technique that I saw / heard about.

Conference Attendance SIGCSE 2015 - Day 1 Morning

It is colder here in Kansas City.  Fortunately, I will only be outside briefly.  Most often I will be networking and continuing my efforts to both become a better teacher, as well as finding an academic job teaching.

This morning, I am focusing on the "Curriculum" track.  I am excited by the three papers in this track, the first looks at research, the second is on systems courses, and the last on parallel computing courses.  Alas, I was in the hallway track and missed the first work.  Perhaps I can find the authors later.

Backward Design: An Integrated Approach to a Systems Curriculum
The goal of systems is "higher level software creation".  Computer Science courses are split into Core Tier 1 and Tier 2 (a term from the ACM 2013 curriculum), where the former are taken by all CS majors and the later are only taken by most or some.  One issue in the old curriculum was that OS also taught C.  In crafting a new curriculum, first establish a vision statement, which can be used in conflict resolution (and also revised).  Establish SMART objectives to prepare and build the assessments.  The results can be found on github.

A Module-based Approach to Adopting the 2013 ACM Curricular Recommendations on Parallel Computing
Parallel computing is important and important for CS graduates to know.  The 2013 ACM Curriculum increased the number of hours that students should take in parallel computing.  Part of the recommendations are to place parallel computing into the curriculum and not just as a course.  Thus parallelism modules are placed throughout the curriculum (perhaps as early as CS1 or CS2).  Find the level of abstraction for a concept and introduce it appropriately.  For example, Amdahl's Law in CS1 versus cache coherence in senior-level class.  5 modules of parallelism were established, which have equivalences with the ACM.  Each course in the curriculum may have 1 or more modules, which then teaches and reinforces the topics.  Even after adding these modules, there has continued to be incremental development and revisions, which have improved student outcomes.  The key take away is that it is possible to introduce these recommendations without completely rewriting the curriculum.

In the afternoon, I will be standing with my poster -  Using Active Learning Techniques in Mixed Undergraduate / Graduate Courses.  Later I will post updates from my afternoon.