A recent ACM article, C is Not a Low-Level Language, argues that for all of our impressions that C is close to hardware, it is not actually "low-level". The argument is as follows, C was written for the PDP-11 architecture and at the time, it was a low-level language. As architectures have evolved, C's machine model has diverged from hardware, which has forced processor design to add new features to attain good performance with C, such as superscalar for ILP and extensive branch prediction.
Processors must also maintain caches to hide the memory latency, which require significant logic to maintain coherence and the illusion that the memory is shared between the threads of a process. Furthermore, the compiler is also called upon to find optimization opportunities that may be unsound and definitely require programmer years to implement.
The author repeatedly contrasts with GPUs, while noting that they require very specific problems, or "at the expense of requiring explicitly parallel programs". If you were not keeping track, a GPU requires thousands of threads to match the CPU's performance. The author calls for, "A processor designed purely for speed, not for a compromise between speed and C support, would likely support large numbers of threads, have wide vector units, and have a much simpler memory model." Which generally sounds like the GPU design.
I appreciate the callouts to C's shortcomings, which it certainly has. The notion that C has driven processor design is odd, yet it does reflect the fact that processors are designed to run current programs fast. And with the programs being written in either C or a language built on C, that forces many programs into particular patterns. I even spent some time in my PhD studies considering a version of this problem: how do you design a new "widget" for the architecture if no programs are designed for widgets to be available?
All to say, I think C is a low-level language, and while its distance from hardware may be growing, there is nothing else beneath it. This is a gap that needs to be addressed, and by a language that has explicit parallel support.
A discussion of how to do Computer Science well, particularly writing code and architecting program solutions.
Showing posts with label programming languages. Show all posts
Showing posts with label programming languages. Show all posts
Friday, September 14, 2018
Wednesday, November 22, 2017
Review: Languages and Code Quality in GitHub
The study is several years old, but was recently reprinted in the Communications of the ACM. In it, they mined GitHub data for active open source projects, collecting the defect and development rates. They classified the defects according to their type, and the development language according to their feature. And they found that language choice matters, marginally. Some types of bug are far more common, such as memory management bugs in C / C++. Functional languages have the lowest rate; however, this analysis is only based on the commit history and does not also analyze development time, or differences in programmers. So the takeaway is that language features do matter, but programmers just write buggy code.
Tuesday, October 24, 2017
PhD Defense - Low-level Concurrent Programming Using the Relaxed Memory Calculus
Today, I went to Michael Sullivan's thesis defense, who passed. The work was at a delightful intersection of my interests.
We want better (more usable, etc) semantics for low-level operations, those below the std::atomic<> and similar designs. Perhaps this is achievable with ordering constraints. Given the following simple example, what constraints are required?
int data, flag;
void send(int msg) {
data = msg;
flag = 1;
}
int recv() {
while (!flag) continue;
return data;
}
Two constraints: data ("visible") before flag, flag ("executed") before data. These constraints are explicitly programmer-specified, and that it is contended that this is practical.
rmc::atomic<T> - a variable that can be concurrently accessed
L(label, expr) - labels an expression
VEDGE and XEDGE - specify orders between labeled expressions, effectively V is write visibility ordering and X is execution of read ordering
rmc::push() or PEDGE - Pushes have a total order, and provide orderings between reads and writes which is not possible with just V and X.
In more advanced space, do we need to add constraints to spinlock_lock and spinlock_unlock? Let's add two special labels: pre, post. These serve for interface boundaries to denote that everything has executed before this point, or is visible.
Next problem is loop iterations. Do the constraints need to be within a single iteration or constraining every iteration? Extend the order specifiers, so in the following, the ordering constraint is just within the iteration, whereas the constraint outside the iteration (without "_HERE") is also between the iterations.
for (i = 0; i < 2; i++) {
VEDGE_HERE(before, after);
L(before, x = i);
L(after, y = i + 10);
}
Code extends LLVM and is on GitHub. The compiler takes the RMC extensions and puts the appropriate fence instructions into the IR, and then the existing compiler lowers this to assembly. The compiler uses an SMT solver to determine the minimal set of locations that need the necessary fences (or other instructions). Then in lowering, the lowering to assembly can better take advantage of the specific constraints required. Overall, the performance is better than the C++11 model on ARMv7, Power, and comparable on ARMv8. I suspect that x86's TSO model is not as interesting for finding performance benefits.
Usable / practical - Can Seqlocks Get Along With Programming Language Memory Models? argues that C++11 would require acquire semantics on unlock. Here it is stated that RMC is much more straightforward. Further, students in 15-418 found gains from RMC versus the C11 model.
Other future items include the exploration of whether there are additional consistency instructions that might provide a better nuance for the compiler to inform hardware about required orderings. Recall that the coarsest grained instruction is the full memory fence.
We want better (more usable, etc) semantics for low-level operations, those below the std::atomic<> and similar designs. Perhaps this is achievable with ordering constraints. Given the following simple example, what constraints are required?
int data, flag;
void send(int msg) {
data = msg;
flag = 1;
}
int recv() {
while (!flag) continue;
return data;
}
Two constraints: data ("visible") before flag, flag ("executed") before data. These constraints are explicitly programmer-specified, and that it is contended that this is practical.
rmc::atomic<T> - a variable that can be concurrently accessed
L(label, expr) - labels an expression
VEDGE and XEDGE - specify orders between labeled expressions, effectively V is write visibility ordering and X is execution of read ordering
rmc::push() or PEDGE - Pushes have a total order, and provide orderings between reads and writes which is not possible with just V and X.
In more advanced space, do we need to add constraints to spinlock_lock and spinlock_unlock? Let's add two special labels: pre, post. These serve for interface boundaries to denote that everything has executed before this point, or is visible.
Next problem is loop iterations. Do the constraints need to be within a single iteration or constraining every iteration? Extend the order specifiers, so in the following, the ordering constraint is just within the iteration, whereas the constraint outside the iteration (without "_HERE") is also between the iterations.
for (i = 0; i < 2; i++) {
VEDGE_HERE(before, after);
L(before, x = i);
L(after, y = i + 10);
}
Code extends LLVM and is on GitHub. The compiler takes the RMC extensions and puts the appropriate fence instructions into the IR, and then the existing compiler lowers this to assembly. The compiler uses an SMT solver to determine the minimal set of locations that need the necessary fences (or other instructions). Then in lowering, the lowering to assembly can better take advantage of the specific constraints required. Overall, the performance is better than the C++11 model on ARMv7, Power, and comparable on ARMv8. I suspect that x86's TSO model is not as interesting for finding performance benefits.
Usable / practical - Can Seqlocks Get Along With Programming Language Memory Models? argues that C++11 would require acquire semantics on unlock. Here it is stated that RMC is much more straightforward. Further, students in 15-418 found gains from RMC versus the C11 model.
Other future items include the exploration of whether there are additional consistency instructions that might provide a better nuance for the compiler to inform hardware about required orderings. Recall that the coarsest grained instruction is the full memory fence.
Friday, March 4, 2016
Conference Attendance SIGCSE 2016 - Day 2
After lunch when we are all in food comas, let's attend the best paper talk!
A Multi-institutional Study of Peer Instruction in Introductory Computing -
This study followed 7 instructors across different institutions as they used peer instruction. This showed that both the instruction is generally recognized as valuable, while also touching on routes in which it can go awry. Tell students why this technique is being used and what it's effect. Hard questions are good questions to ask, as students will discuss and learn from the question. This requires that questions are graded for participation and not *correctness*. Possible questions and material for peer instruction is available.
Development of a Concept Inventory for Computer Science Introductory Programming -
A concept inventory is a set of questions that carefully tease out student misunderstandings and misconceptions. Take the exams and identify both the learning objective and the misconception that results in incorrect answers.
int addFiveToNumber(int n)
{
int c = 0;
// Insert line here
return c;
}
int main(int argc, char** argv)
{
int x = 0;
x = addFiveToNumber(x);
printf("%d\n", x);
return 0;
}
a) scanf("%d", &n);
b) n = n + 5;
c) c = n + 5;
d) x = x + 5;
Each incorrect answer illustrates a different misconception. For example, input must come from the keyboard. Or variables are passed by reference.
Overall, this study illustrated how the concept inventory was developed, but not the impact of having it, or what it showed in the students and their learning.
Uncommon Teaching Languages - (specifically in intro courses)
An interesting effect of using an uncommon language in an introductory course is that the novices and experts have similar skills. Languages should be chosen to minimize churn, otherwise students feel that they haven't mastered any languages. And related to this point, languages also exist in an institutional ecosystem. Furthermore, we want to minimize the keywords / concepts required for a simple program. A novice will adopt these keywords, but they also are "magic" and arcane. And then how long are the programs, as we want novices to only have to write short code to start.
I also attended the SIGCSE business meeting and then the NCWIT reception. I have gone to NCWIT every year at SIGCSE, as I want to know what I should do (or not do) to not bias anyone's experience in Computer Science.
A Multi-institutional Study of Peer Instruction in Introductory Computing -
This study followed 7 instructors across different institutions as they used peer instruction. This showed that both the instruction is generally recognized as valuable, while also touching on routes in which it can go awry. Tell students why this technique is being used and what it's effect. Hard questions are good questions to ask, as students will discuss and learn from the question. This requires that questions are graded for participation and not *correctness*. Possible questions and material for peer instruction is available.
Development of a Concept Inventory for Computer Science Introductory Programming -
A concept inventory is a set of questions that carefully tease out student misunderstandings and misconceptions. Take the exams and identify both the learning objective and the misconception that results in incorrect answers.
int addFiveToNumber(int n)
{
int c = 0;
// Insert line here
return c;
}
int main(int argc, char** argv)
{
int x = 0;
x = addFiveToNumber(x);
printf("%d\n", x);
return 0;
}
a) scanf("%d", &n);
b) n = n + 5;
c) c = n + 5;
d) x = x + 5;
Each incorrect answer illustrates a different misconception. For example, input must come from the keyboard. Or variables are passed by reference.
Overall, this study illustrated how the concept inventory was developed, but not the impact of having it, or what it showed in the students and their learning.
Uncommon Teaching Languages - (specifically in intro courses)
An interesting effect of using an uncommon language in an introductory course is that the novices and experts have similar skills. Languages should be chosen to minimize churn, otherwise students feel that they haven't mastered any languages. And related to this point, languages also exist in an institutional ecosystem. Furthermore, we want to minimize the keywords / concepts required for a simple program. A novice will adopt these keywords, but they also are "magic" and arcane. And then how long are the programs, as we want novices to only have to write short code to start.
I also attended the SIGCSE business meeting and then the NCWIT reception. I have gone to NCWIT every year at SIGCSE, as I want to know what I should do (or not do) to not bias anyone's experience in Computer Science.
Tuesday, December 1, 2015
Course Design Series (Post 3 of N) - Features versus Languages
One problem I had in preparing the curriculum for this fall is how to balance teaching programming languages versus the features that exist in programming languages. Some analogs to my course have generally been surveys of programming languages, for example - Michael Hewner's course. In such a course, study is focused strongly on learning a diverse set of languages, particularly pulling from logic and functional programming paradigms.
In general, the problem is that the each feature needs examples from programming languages to illustrate how it is used; however, many of these languages have never been previously taught to the students. Therefore, I could spend the first month teaching the basics of Ada, Fortran, Pascal, et cetera. But in doing this, essentially the class starts as a language class. I had considered this approach; however, the non-survey courses and the textbooks do not begin with the languages. These examples all begin with the features. Furthermore, I have taken the further approach to avoid focusing on specific syntax of languages and instead devote my time to teaching features and design.
Having then accepted that the textbooks had a purpose and were rational in design, I followed their structure. And in retrospect I can see that having upper-level students provides a base of knowledge about programming languages such that the need to cover specifics is avoided. I have still taken some class periods to delve into specific languages; however, this is the exception rather than a rule. I do note that in the future, I would spend some time teaching / requiring specific languages. Without this requirement, I have been faced with grading a myriad of languages and find myself unable to assign problems / questions based on specific languages.
In general, the problem is that the each feature needs examples from programming languages to illustrate how it is used; however, many of these languages have never been previously taught to the students. Therefore, I could spend the first month teaching the basics of Ada, Fortran, Pascal, et cetera. But in doing this, essentially the class starts as a language class. I had considered this approach; however, the non-survey courses and the textbooks do not begin with the languages. These examples all begin with the features. Furthermore, I have taken the further approach to avoid focusing on specific syntax of languages and instead devote my time to teaching features and design.
Having then accepted that the textbooks had a purpose and were rational in design, I followed their structure. And in retrospect I can see that having upper-level students provides a base of knowledge about programming languages such that the need to cover specifics is avoided. I have still taken some class periods to delve into specific languages; however, this is the exception rather than a rule. I do note that in the future, I would spend some time teaching / requiring specific languages. Without this requirement, I have been faced with grading a myriad of languages and find myself unable to assign problems / questions based on specific languages.
Friday, October 2, 2015
SCS Lecture Series: Solver-Aided Declarative Programming
Intersection of Programming Languages, Databases, and analytics (Machine Learning, etc)
Businesses are big systems that can then be modeled: financial, data, supply chain, consumer demand, etc. Simple models can be analytical formulas or spreadsheets prepared by the domain expert. Or an enterprise system of database + Java layer + .... Consider the thousands or millions of SKUs plus thousands of stores. The goal is that the domain expert can build and maintain the model.
For example, one client had 234 billion weekly forecasts (500k skus x 9000 stores x 52 weeks). Consider the impact of BOGO deals and placing items on endcaps. How much does this improve sales? Accuracy of predictions had lower error than competing forecasts. Resulted in millions of dollars in savings by optimizing inventory. And benefit to the customer by using cloud computing versus requiring the customer to buy the necessary computing resources. Customers trust the cloud more to be secure as companies such as Target, Home Depot, T-Mobile, ... have suffered data breaches. Who do you think has more resources to secure the data: google or Target?
Recall that Excel is the world's most popular IDE with the most popular programming language. Thus the audience for this work are the business people, not the computer scientists. So have a programming language of LogiQL, inheriting from datalog. Language is not Turing-complete, but this provides many valuable analyses that would otherwise be undecidable. Provide vastly more efficient clique finding in graphs.
Integrity constraints give possible states for the data and the relation. Therefore, the database knows what is possible in its facts. Or with weighted constraints, which facts violate the fewest constraints. Include an operator (ESO - existential second order) such that rather than a fact exists. ESO is quantified by a relation existing. For example, ESO can be invoked to find a valid color relation for graph coloring.
Businesses are big systems that can then be modeled: financial, data, supply chain, consumer demand, etc. Simple models can be analytical formulas or spreadsheets prepared by the domain expert. Or an enterprise system of database + Java layer + .... Consider the thousands or millions of SKUs plus thousands of stores. The goal is that the domain expert can build and maintain the model.
For example, one client had 234 billion weekly forecasts (500k skus x 9000 stores x 52 weeks). Consider the impact of BOGO deals and placing items on endcaps. How much does this improve sales? Accuracy of predictions had lower error than competing forecasts. Resulted in millions of dollars in savings by optimizing inventory. And benefit to the customer by using cloud computing versus requiring the customer to buy the necessary computing resources. Customers trust the cloud more to be secure as companies such as Target, Home Depot, T-Mobile, ... have suffered data breaches. Who do you think has more resources to secure the data: google or Target?
Recall that Excel is the world's most popular IDE with the most popular programming language. Thus the audience for this work are the business people, not the computer scientists. So have a programming language of LogiQL, inheriting from datalog. Language is not Turing-complete, but this provides many valuable analyses that would otherwise be undecidable. Provide vastly more efficient clique finding in graphs.
Integrity constraints give possible states for the data and the relation. Therefore, the database knows what is possible in its facts. Or with weighted constraints, which facts violate the fewest constraints. Include an operator (ESO - existential second order) such that rather than a fact exists. ESO is quantified by a relation existing. For example, ESO can be invoked to find a valid color relation for graph coloring.
Monday, August 17, 2015
Course Design Series (Post 2 of N): Choosing a Textbook
Having now read both Programming Language Pragmatics, Third Edition and Concepts of Programming Languages (11th Edition), I have settled on the former as my textbook for the fall. I do not find either book ideally suited, and I wish that the Fourth Edition was being released this summer and not in November, which is why it still hasn't arrived.
For the choice of which textbook to use, it was "Concepts" to lose and having 11 editions, the text should also be better revised. I dislike reading the examples in a book and questioning how the code would compile. Beyond which, the examples felt quaint and contrived.
(edit) For example, I was reviewing the material on F# and copied in an example from the text:
let rec factorial x =
if x <= 1 then 1
else n * factorial(n-1)
Does anyone else notice that the function parameter is x on the first two lines and n on the last?!
Before the Concepts book is written off entirely, there are many valuable aspects. I enjoyed reading about the history of programming languages, especially for exposing me to Plankalkül. The work also took a valuable track in the subject by regularly looking at the trade-offs between different designs and features. This point certainly helped inform my teaching of the material.
Fundamentally, when I looked at the price of the two textbooks, the benefits of using the newer Concepts textbook could not outweigh the nearly doubled pricetag. Most of the positive points are small things and can be covered as addendums to the material.
(FCC note - Concepts of Programming Languages was provided free to me by the publisher.)
For the choice of which textbook to use, it was "Concepts" to lose and having 11 editions, the text should also be better revised. I dislike reading the examples in a book and questioning how the code would compile. Beyond which, the examples felt quaint and contrived.
(edit) For example, I was reviewing the material on F# and copied in an example from the text:
let rec factorial x =
if x <= 1 then 1
else n * factorial(n-1)
Does anyone else notice that the function parameter is x on the first two lines and n on the last?!
Before the Concepts book is written off entirely, there are many valuable aspects. I enjoyed reading about the history of programming languages, especially for exposing me to Plankalkül. The work also took a valuable track in the subject by regularly looking at the trade-offs between different designs and features. This point certainly helped inform my teaching of the material.
Fundamentally, when I looked at the price of the two textbooks, the benefits of using the newer Concepts textbook could not outweigh the nearly doubled pricetag. Most of the positive points are small things and can be covered as addendums to the material.
(FCC note - Concepts of Programming Languages was provided free to me by the publisher.)
Wednesday, June 17, 2015
Conference Attendance FCRC - Day 5 - Plenary Summary
Plenary Talk today, which pulls together all of the conference attendees. Sunday's talk was based in databases, with Michael Stonebraker speaking on his Turing-award winning work. Monday's talk discussed interdisciplinary work, primarily centered in CS theory, and was given by Andrew Yao (a prior Turing Award winner). On Tuesday, Olivier Temam discussed neural networks in hardware, which focused on his work and efforts to better model or mimic the capabilities of the brain.
The F# Path to Relaxation -
There are opportunities to introduce new work toward relaxing and improving. Or perhaps create opposing camps. Thesis <-> Antithesis ==> synthesis. Or Functional <=> Interop. Back in 2003, functional languages were isolated, non-interoperable, using their own VMs. F# (along with Scala, Swift, ...) instead seeks to have an exosystem, being the external industry-standard runtimes. Another tension is between Enterprise and Openness. So F# is open and cross-platform. Tools are available for Android and iOS, as well as packages for Linux.
Functional <=> Objects
Thus embrace objects, without being object-oriented. Some cases in the cross-product of the expected features for objects and functions requires particular care for synthesis.
Circularities and Modularity in the Wild
Lambdas, generics, etc are clearly being embraced in modern language design. However, circular type dependencies are unfortunately also widely present. Languages need to enforce acyclicity.
Pattern Matching <=> Abstraction
How does the language support the functional concept of pattern matching, when you want to include type abstraction? Alas, the speaker skipped the solution quickly.
Code <=> Data
Most development is to providing tools for the information revolution. There is exponential growth in Open APIs for accessing data from the internet. This data then comes with dynamic types, where the types are only known once the data (or schema) has been accessed. The type creation can also enable blending code for other languages into the F# environment. For example, the support can allow opening csv or json files and having types for the data. This feature is, by far, the most exciting and interesting of the presentation. Not quite worth the price of admission, but clearly a great development.
Applied PL design comes from the synthesis at the heart of these contradictions. This tension also is part of the proliferation of languages.
The F# Path to Relaxation -
There are opportunities to introduce new work toward relaxing and improving. Or perhaps create opposing camps. Thesis <-> Antithesis ==> synthesis. Or Functional <=> Interop. Back in 2003, functional languages were isolated, non-interoperable, using their own VMs. F# (along with Scala, Swift, ...) instead seeks to have an exosystem, being the external industry-standard runtimes. Another tension is between Enterprise and Openness. So F# is open and cross-platform. Tools are available for Android and iOS, as well as packages for Linux.
Functional <=> Objects
Thus embrace objects, without being object-oriented. Some cases in the cross-product of the expected features for objects and functions requires particular care for synthesis.
Circularities and Modularity in the Wild
Lambdas, generics, etc are clearly being embraced in modern language design. However, circular type dependencies are unfortunately also widely present. Languages need to enforce acyclicity.
Pattern Matching <=> Abstraction
How does the language support the functional concept of pattern matching, when you want to include type abstraction? Alas, the speaker skipped the solution quickly.
Code <=> Data
Most development is to providing tools for the information revolution. There is exponential growth in Open APIs for accessing data from the internet. This data then comes with dynamic types, where the types are only known once the data (or schema) has been accessed. The type creation can also enable blending code for other languages into the F# environment. For example, the support can allow opening csv or json files and having types for the data. This feature is, by far, the most exciting and interesting of the presentation. Not quite worth the price of admission, but clearly a great development.
Applied PL design comes from the synthesis at the heart of these contradictions. This tension also is part of the proliferation of languages.
Conference Attendance FCRC - Day 4 - PLDI
PLDI starts off this morning with Concurrency. As a student volunteer, I worked this session and was limited as to what I could note about the content itself.
Composing Concurrency Control - Introducing more diverse and finer-grained locking mechanisms. The tool works to develop a locking strategy that will guarantee serializability, abort-safety, opacity, and deadlock-freedom. It particularly works to integrate both locking schemes as well as transactional memory.
In the afternoon, I can dive into the semantics of the C language.
A Formal C Memory Model Supporting Integer-Pointer Casts - What optimizations are possible in the presence of pointers, pointer arithmetic, and integer-pointer casts? For example, can constants be propagated or is their location potentially targetable by a pointer? Other optimizations are explored in their paper. In practice, as code can generate arbitrary addresses, how can the compiler reason about any specific location in memory.
Defining the Undefinedness of C - Extending their prior work that gave semantics to defined behavior of C programs, which required doubling the rules to describe the semantic behavior. Fundamentally, any instance of undefined behavior that will be definitely encountered in an execution will invalidate that execution. For example, dividing by zero after a printf is valid to crash before the printf. The following code example is also undefined.
Composing Concurrency Control - Introducing more diverse and finer-grained locking mechanisms. The tool works to develop a locking strategy that will guarantee serializability, abort-safety, opacity, and deadlock-freedom. It particularly works to integrate both locking schemes as well as transactional memory.
In the afternoon, I can dive into the semantics of the C language.
A Formal C Memory Model Supporting Integer-Pointer Casts - What optimizations are possible in the presence of pointers, pointer arithmetic, and integer-pointer casts? For example, can constants be propagated or is their location potentially targetable by a pointer? Other optimizations are explored in their paper. In practice, as code can generate arbitrary addresses, how can the compiler reason about any specific location in memory.
Defining the Undefinedness of C - Extending their prior work that gave semantics to defined behavior of C programs, which required doubling the rules to describe the semantic behavior. Fundamentally, any instance of undefined behavior that will be definitely encountered in an execution will invalidate that execution. For example, dividing by zero after a printf is valid to crash before the printf. The following code example is also undefined.
return (x = 1) + (x = 2);Many of these cases are dependent on runtime behavior, and therefore a tool that can help identify them is valuable.
Monday, June 15, 2015
Conference Attendance FCRC - Day 3 - PLDI / ISCA
PLDI itself began this morning and after the welcome, we had three distinguished papers. I am excited that two of these works focused on code performance and compilers, rather than higher-level programming language issues:
Automatically Improving the Accuracy of Floating Point Expressions - How do you address rounding error in your code? Use formal numeric methods an expert can reduce the errors. But rather than be an expert, they wrote a tool to use heuristics to apply these methods. For example, what error do you have when evaluating the quadratic formula. Based on just the value for b, there are different expressions that have much lower error.
The tool, Herbie, estimates the accuracy of the expression and then attempts to use algebraic transformations (from a database of 120 rules). Having generated many candidate expressions, the tool then selects using dynamic programming an appropriate set of expressions across the input space. First, it matches the example cases from the Hamming's Numeric Methods book. And furthermore has found bugs in existing projects.
Diagnosing Type Errors with Class - SHErrLoc works to identify the likely cause of type errors. Expressions are given constraints. These constraints form a graph, which is analyzed for failing paths in the graph. The tool then attempts to localize the failure and identify the minimal change to the constraints to satisfy the graph. Even though it is not Haskell specific, it is more accurate at detecting type errors in Haskell programs than related work.
Provably Correct Peephole Optimizations with Alive - Compilers are buggy. For example, LLVM's InstCombine is an LLVM pass that exploits the LLVM IR to improve performance, which contains many hand-rolled transformations. Propose a DSL that describes Peephole Optimizations, where the the DSL is basically a simplified LLVM IR annotated with preconditions for the transformation. Then the expression describing the transformation is passed through constraint checkers to verify it is correct. And then generate C++ code for that transformation.
Correctness of the expression must not introduce new undefined behaviors, still produces the same result, and properly updates the memory state. Initially proved the optimizations in InstCombine correct or identified bugs, and eventually could replace the pass with the generated version. Furthermore, Alive was able to strengthen the post-conditions for many instructions (for example, identifying whether an operation will overflow).
In the afternoon, I was visiting the other side of the conference center with ISCA. One paper of note there:
A Scalable Processing-in-Memory Accelerator for Parallel Graph Processing - They showed that (somehow, as I missed this point) integrating the simple core closer to the memory system, they pushed past the memory bandwidth of HMC (640GB/s) and instead to about 2.3TB/s. They focused on two pieces: an updated programming model for graphs and a prefetching system within their cores. The model introduced async remote procedure calls that are sent to the Tesseract core near the data. These messages accumulate in a queue until either a barrier or the queue is full. While they accumulate, the prefetcher is requesting the appropriate data so when the function fires, the data is available. The prefetcher is able to operate on the two separate streams: the local processing that is sequential and generating the remote requests, and then the remote requests received at this node.
Automatically Improving the Accuracy of Floating Point Expressions - How do you address rounding error in your code? Use formal numeric methods an expert can reduce the errors. But rather than be an expert, they wrote a tool to use heuristics to apply these methods. For example, what error do you have when evaluating the quadratic formula. Based on just the value for b, there are different expressions that have much lower error.
The tool, Herbie, estimates the accuracy of the expression and then attempts to use algebraic transformations (from a database of 120 rules). Having generated many candidate expressions, the tool then selects using dynamic programming an appropriate set of expressions across the input space. First, it matches the example cases from the Hamming's Numeric Methods book. And furthermore has found bugs in existing projects.
Diagnosing Type Errors with Class - SHErrLoc works to identify the likely cause of type errors. Expressions are given constraints. These constraints form a graph, which is analyzed for failing paths in the graph. The tool then attempts to localize the failure and identify the minimal change to the constraints to satisfy the graph. Even though it is not Haskell specific, it is more accurate at detecting type errors in Haskell programs than related work.
Provably Correct Peephole Optimizations with Alive - Compilers are buggy. For example, LLVM's InstCombine is an LLVM pass that exploits the LLVM IR to improve performance, which contains many hand-rolled transformations. Propose a DSL that describes Peephole Optimizations, where the the DSL is basically a simplified LLVM IR annotated with preconditions for the transformation. Then the expression describing the transformation is passed through constraint checkers to verify it is correct. And then generate C++ code for that transformation.
Correctness of the expression must not introduce new undefined behaviors, still produces the same result, and properly updates the memory state. Initially proved the optimizations in InstCombine correct or identified bugs, and eventually could replace the pass with the generated version. Furthermore, Alive was able to strengthen the post-conditions for many instructions (for example, identifying whether an operation will overflow).
In the afternoon, I was visiting the other side of the conference center with ISCA. One paper of note there:
A Scalable Processing-in-Memory Accelerator for Parallel Graph Processing - They showed that (somehow, as I missed this point) integrating the simple core closer to the memory system, they pushed past the memory bandwidth of HMC (640GB/s) and instead to about 2.3TB/s. They focused on two pieces: an updated programming model for graphs and a prefetching system within their cores. The model introduced async remote procedure calls that are sent to the Tesseract core near the data. These messages accumulate in a queue until either a barrier or the queue is full. While they accumulate, the prefetcher is requesting the appropriate data so when the function fires, the data is available. The prefetcher is able to operate on the two separate streams: the local processing that is sequential and generating the remote requests, and then the remote requests received at this node.
Wednesday, May 13, 2015
Course Design Series (Post 1 of N): Why Study Programming Languages
Obviously, there are some practitioners and researchers who base their living on programming language design. But what of the rest of us? The Communications of the ACM ran a short article on why: Teach foundational language principles. Language design is increasingly focused on programmer productivity and correctness. As programmers, are we aware of the new features and programming paradigms?
Particularly, let's look at three aspects of programming languages: contracts, functional languages, and type systems. By introducing each to budding programmers, we improve their habits, just as presently I include comments and other style components in project grades. First, contracts provide a well defined mechanism for specifying the requirements of each component in a system. Not just commenting each component and interface, but doing so in a manner that permits the compiler / runtime system to verify and enforce it.
Functional languages are well known, and whether or not you may ever use one, they teach valuable techniques and mental models regarding programming. A programmer should use pure and side-effect free procedures, rather than interleaving logic across different components. Other practices, such as unit testing, also trend toward clean interfaces; however, being forced to obey these rules via the underlying language is great practice.
Type systems are treated by the authors as a panacea, as they call for language designers to be "educated in the formal foundations of safe programming languages - type systems." Panacea or not, reasoning about functionality within the bounds of types leads to code that is clearer and more maintainable. Even in a weak language, such as C, one can use enums rather than everything being "int".
As these aspects are being increasingly used in various forms in current language design, programmers need to be knowledgeable of them in order to be effective and use the languages to their full potential. It is therefore incumbent on me, and other educators, to appropriately include these aspects when we teach about programming languages.
Which leads me back to writing a course description and learning objectives for my fall course. Maybe later this month once I figure out why one of the textbooks still hasn't arrived.
Particularly, let's look at three aspects of programming languages: contracts, functional languages, and type systems. By introducing each to budding programmers, we improve their habits, just as presently I include comments and other style components in project grades. First, contracts provide a well defined mechanism for specifying the requirements of each component in a system. Not just commenting each component and interface, but doing so in a manner that permits the compiler / runtime system to verify and enforce it.
Functional languages are well known, and whether or not you may ever use one, they teach valuable techniques and mental models regarding programming. A programmer should use pure and side-effect free procedures, rather than interleaving logic across different components. Other practices, such as unit testing, also trend toward clean interfaces; however, being forced to obey these rules via the underlying language is great practice.
Type systems are treated by the authors as a panacea, as they call for language designers to be "educated in the formal foundations of safe programming languages - type systems." Panacea or not, reasoning about functionality within the bounds of types leads to code that is clearer and more maintainable. Even in a weak language, such as C, one can use enums rather than everything being "int".
As these aspects are being increasingly used in various forms in current language design, programmers need to be knowledgeable of them in order to be effective and use the languages to their full potential. It is therefore incumbent on me, and other educators, to appropriately include these aspects when we teach about programming languages.
Which leads me back to writing a course description and learning objectives for my fall course. Maybe later this month once I figure out why one of the textbooks still hasn't arrived.
Thursday, April 2, 2015
Course Design Series (Post 0 of N): A New Course
This fall I will again be Instructor of Record. My appointment is to teach CS 4392 - Programming Languages. This is an unusual situation in that the course has not been taught for over 5 years (last time was Fall 2009). Effectively, the course will have to be designed afresh.
The first step in course design (following L. Dee Fink and McKeachie's Teaching Tips: Chapter 2) is to write the learning objectives using the course description, along with prerequisites and courses that require this one. Let's review what we have:
Course Description: none
Prerequisites:
Undergraduate Semester level CS 2340 (which has the following description)
Object-oriented programming methods for dealing with large programs. Focus on quality processes, effective debugging techniques, and testing to assure a quality product.
Courses depending on this: none
Alright. Now I will turn to the CS curriculum at Georgia Tech. Georgia Tech uses a concept they call "threads", which are sets of related courses. CS 4392 is specifically in the Systems and Architecture thread. This provides several related courses:
CS 4240 - Compilers, Interpreters, and Program Analyzers
Study of techniques for the design and implementation of compilers, interpreters, and program analyzers, with consideration of the particular characteristics of widely used programming languages.
CS 6241 - Compiler Design
Design and implementation of modern compilers, focusing upon optimization and code generation.
CS 6390 - Programming Languages
Design, structure, and goals of programming languages. Object-oriented, logic, functional, and traditional languages. Semantic models. Parallel programming languages.
Finally, the ACM has provided guidelines for the CS curriculum. Not only does this provide possible options for what material I should include, but they have also provided several ACM exemplar courses (c.f., http://www.cs.rochester.edu/ courses/254/fall2013/ and http://courses.cs.washington. edu/courses/cse341/13sp/).
To summarize, if my first step is to write the learning objectives, then I am on step 0: write the course description. In a couple of weeks, I plan on finishing my initial review of potential textbooks as well as the other materials covered above. That will provide me the groundwork for the description and then objectives.
The first step in course design (following L. Dee Fink and McKeachie's Teaching Tips: Chapter 2) is to write the learning objectives using the course description, along with prerequisites and courses that require this one. Let's review what we have:
Course Description: none
Prerequisites:
Undergraduate Semester level CS 2340 (which has the following description)
Object-oriented programming methods for dealing with large programs. Focus on quality processes, effective debugging techniques, and testing to assure a quality product.
Courses depending on this: none
Alright. Now I will turn to the CS curriculum at Georgia Tech. Georgia Tech uses a concept they call "threads", which are sets of related courses. CS 4392 is specifically in the Systems and Architecture thread. This provides several related courses:
CS 4240 - Compilers, Interpreters, and Program Analyzers
Study of techniques for the design and implementation of compilers, interpreters, and program analyzers, with consideration of the particular characteristics of widely used programming languages.
CS 6241 - Compiler Design
Design and implementation of modern compilers, focusing upon optimization and code generation.
CS 6390 - Programming Languages
Design, structure, and goals of programming languages. Object-oriented, logic, functional, and traditional languages. Semantic models. Parallel programming languages.
Finally, the ACM has provided guidelines for the CS curriculum. Not only does this provide possible options for what material I should include, but they have also provided several ACM exemplar courses (c.f., http://www.cs.rochester.edu/
To summarize, if my first step is to write the learning objectives, then I am on step 0: write the course description. In a couple of weeks, I plan on finishing my initial review of potential textbooks as well as the other materials covered above. That will provide me the groundwork for the description and then objectives.
Monday, July 14, 2014
Book Review: The Practice of Programming
(contains Amazon affiliate link)
I recently found, The Practice of Programming (Addison-Wesley Professional Computing Series), sitting on the shelf at my local library. I am generally skeptical when it comes to programming books, and particularly those from different decades, but I trusted the name "Brian Kernighan" so I checked the book out.
And I am so glad that I did. From the first chapter that discussed style, I wanted to read more. And the only reason to ever stop reading was to pull out a computer and put these things into practice. I didn't even mind that it wasn't until chapter 7 that performance was discussed. Still, I will readily acknowledge that I disagree with some of statements in the book. Furthermore, there are some parts of the text that are clearly dated, like discussing current C / C++ standards.
I'd like to conclude with a brief code snippet from the work. This code is part of a serializer / deserializer. Such routines are always a pain to write and particularly if you have many different classes / structs that need them. Thus the authors suggest using vargs and writing a single routine that can handle this for you. Here is the unpack (i.e., deserialize) routine:
/* unpack: unpack packed items from buf, return length */
int unpack(uchar *buf, char *fmt, ...)
{
va_list args;
char *p;
uchar *bp, *pc;
ushort *ps;
ulong *pl;
bp = buf;
va_start(args, fmt);
for (p = fmt; *p != '\0'; p++) {
switch (*p) {
case 'c': /* char */
pc = va_arg(args, uchar*);
*pc = *bp++;
break;
case 's': /* short */
ps = va_arg(args, ushort*);
*ps = *bp++ << 8;
*ps |= *bp++;
break;
case 'l': /* long */
pl = va_arg(args, ulong*);
*pl = *bp++ << 24;
*pl |= *bp++ << 16;
*pl |= *bp++ << 8;
*pl |= *bp++;
default: /* illegal type character */
va_end(args);
return -1;
}
}
va_end(args);
return bp - buf;
}
So now we have a little language for describing the format of the data in the buffer. We invoke unpack with a string like "cscl" and pointers to store the char, short, char and long. Hah! That's it. Anytime we add new types, we just to call the pack / unpack.
Does it matter that the variables are only sequences like "pl" or "bp"? No. Variable names should be meaningful and consistent. "i" can be fine for a loop iterator.
We have given up some performance (*gasp*), but gained in the other parts that matter like readability and maintainability. I plan on using this in my current research (but only the unpack, as my serializers are already highly optimized). All in all, I approve of this book and may even someday require it as a textbook for students.
I recently found, The Practice of Programming (Addison-Wesley Professional Computing Series), sitting on the shelf at my local library. I am generally skeptical when it comes to programming books, and particularly those from different decades, but I trusted the name "Brian Kernighan" so I checked the book out.
And I am so glad that I did. From the first chapter that discussed style, I wanted to read more. And the only reason to ever stop reading was to pull out a computer and put these things into practice. I didn't even mind that it wasn't until chapter 7 that performance was discussed. Still, I will readily acknowledge that I disagree with some of statements in the book. Furthermore, there are some parts of the text that are clearly dated, like discussing current C / C++ standards.
I'd like to conclude with a brief code snippet from the work. This code is part of a serializer / deserializer. Such routines are always a pain to write and particularly if you have many different classes / structs that need them. Thus the authors suggest using vargs and writing a single routine that can handle this for you. Here is the unpack (i.e., deserialize) routine:
/* unpack: unpack packed items from buf, return length */
int unpack(uchar *buf, char *fmt, ...)
{
va_list args;
char *p;
uchar *bp, *pc;
ushort *ps;
ulong *pl;
bp = buf;
va_start(args, fmt);
for (p = fmt; *p != '\0'; p++) {
switch (*p) {
case 'c': /* char */
pc = va_arg(args, uchar*);
*pc = *bp++;
break;
case 's': /* short */
ps = va_arg(args, ushort*);
*ps = *bp++ << 8;
*ps |= *bp++;
break;
case 'l': /* long */
pl = va_arg(args, ulong*);
*pl = *bp++ << 24;
*pl |= *bp++ << 16;
*pl |= *bp++ << 8;
*pl |= *bp++;
default: /* illegal type character */
va_end(args);
return -1;
}
}
va_end(args);
return bp - buf;
}
So now we have a little language for describing the format of the data in the buffer. We invoke unpack with a string like "cscl" and pointers to store the char, short, char and long. Hah! That's it. Anytime we add new types, we just to call the pack / unpack.
Does it matter that the variables are only sequences like "pl" or "bp"? No. Variable names should be meaningful and consistent. "i" can be fine for a loop iterator.
We have given up some performance (*gasp*), but gained in the other parts that matter like readability and maintainability. I plan on using this in my current research (but only the unpack, as my serializers are already highly optimized). All in all, I approve of this book and may even someday require it as a textbook for students.
Tuesday, March 18, 2014
Turing Complete x86 Mov
Here is a small work that showed the power (or perhaps horror) of modern assembly languages. I've told people before that an instruction to "subtract and branch if negative" is sufficient for all programming. But never would have I imagined that x86's mov is also sufficient. Until you think about what Intel lets you do:
- Set the instruction pointer
- Compute arithmetic expressions via address calculation
- Compare two values by using their corresponding locations
Monday, July 25, 2011
Book Review: Masterminds of Programming
Several years ago I read a couple of interesting interviews with language designers, so my curiously was piqued by Masterminds of Programming: Conversations with the Creators of Major Programming Languages (Theory in Practice (O'Reilly))
. While the title is pretentious, the premise was interesting in that here are 17 well-used languages, so what was the designer thinking? Why did he make the choices that he did?
Alas, it was not quite to be. Interesting yes, but the discussions were sometimes more general involving all of computer science. Yet nearly every interview had memorable lines and claims; a few of which I shall reproduce here:
"C is a reasonably good language for compilers to generate, but the idea that humans beings should program in it is completely absurd." - Bertrand Meyer, Eiffel
"One of our programming languages guys proposed a competition in which people would program the same program in all three of those [languages] and we'd see .... It turned out the only thing we figured out is it all depended on where the brightest programmer went...." - Charles Geschke, PostScript
"A good programmer writes good code quickly. Good code is correct, compact and readable. 'Quickly' means hours to days."
and
"An operating system does absolutely nothing for you. As long as you had something - a subroutine called a disk driver, a subroutine called some kind of communication support, in the modern world, it doesn't do anything else." - Chuck Moore, FORTH
"I do recommend [C++] and not everybody is reluctant. In fact, I don't see much reluctance in [system software or embedded systems] beyond the natural reluctance to try something new in established organizations. Rather, I see steady and significant growth in C++ use."
and
"I have never seen a program that could be written better in C than in C++. I don't think such a program could exist." - Bjarne Stroustrup, C++
So now I go to Amazon to remove the book from the list to read, and record a score of 3 out of 5.
Alas, it was not quite to be. Interesting yes, but the discussions were sometimes more general involving all of computer science. Yet nearly every interview had memorable lines and claims; a few of which I shall reproduce here:
"C is a reasonably good language for compilers to generate, but the idea that humans beings should program in it is completely absurd." - Bertrand Meyer, Eiffel
"One of our programming languages guys proposed a competition in which people would program the same program in all three of those [languages] and we'd see .... It turned out the only thing we figured out is it all depended on where the brightest programmer went...." - Charles Geschke, PostScript
"A good programmer writes good code quickly. Good code is correct, compact and readable. 'Quickly' means hours to days."
and
"An operating system does absolutely nothing for you. As long as you had something - a subroutine called a disk driver, a subroutine called some kind of communication support, in the modern world, it doesn't do anything else." - Chuck Moore, FORTH
"I do recommend [C++] and not everybody is reluctant. In fact, I don't see much reluctance in [system software or embedded systems] beyond the natural reluctance to try something new in established organizations. Rather, I see steady and significant growth in C++ use."
and
"I have never seen a program that could be written better in C than in C++. I don't think such a program could exist." - Bjarne Stroustrup, C++
So now I go to Amazon to remove the book from the list to read, and record a score of 3 out of 5.
Saturday, March 26, 2011
Functional vs Imperative Programming
I was recently provided a link to another "programming" blog. The blog, Existential Type, is written by a professor from my undergrad institution, Carnegie Mellon. In it, he appears to be chronicling his efforts of teaching functional programming to freshmen computer science students. I have been deeply engrossed in reading the posts thus far and they have revived knowledge of past debates in language choice, etc. You may look forward to several future posts delving deeper into this issue. But today is Saturday and I have more research to do.
Thursday, September 9, 2010
Problem Solving via Programming Languages
As an interlude to the OO series, I want to clarify something about language preference.
BASIC, C, C++, C#, IA64, Java, Lisp, MIPS, ML / OCAML, Perl, Python, and X86 / X64 -- I've written executable code in them all (some far more than others). While C dominates my coding experience, and is my general preference, I recognize that sometimes another language provides better support. Usually, this support is primarily a reduction in development time, but I'd like to delve a little deeper into why I choose three particular languages: C, C#, and Perl.
C -- "Machine independent assembly"
When I chose to write C code, I'm often thinking of precisely how a computer will execute the high-level solution that I've sketched. Usually, given a problem, I am decomposing it into executable bits, like looping through a list, building a tree just-so, etc. I know what I want, and if I'm wrong then its my foot that I'm shooting.
C# -- "Rapid application development"
I'm only beginning to use C# regularly, but I'm finding it very useful for achieving simple tasks, particularly those that require a UI. .NET is providing an extensive collection of support libraries, so that loading graphics, interacting with them, etc is mostly a matter of connecting the pieces together. To me C# is called for when my problem is UI-centric: click the button and an image appears....
Perl -- "Text, files and strings"
My Perl use has been declining over the years, not for a dislike of the language, but rather a shortage of problems that call for its use. Perl is my tool of choice when I need to work with lots of text and files. I've used it to write everything from copy files between directions and rename to parse MBs of raw data and compute statistics for particular results of interest. (The later eventually was rewritten in C for speed and portability).
I'm always thinking that one of these days I'll try learning more about a functional language. However, I quickly fall into the trap of thinking procedurally. "I know you're on an x86, so just let me write my assembly and we can be done."
BASIC, C, C++, C#, IA64, Java, Lisp, MIPS, ML / OCAML, Perl, Python, and X86 / X64 -- I've written executable code in them all (some far more than others). While C dominates my coding experience, and is my general preference, I recognize that sometimes another language provides better support. Usually, this support is primarily a reduction in development time, but I'd like to delve a little deeper into why I choose three particular languages: C, C#, and Perl.
C -- "Machine independent assembly"
When I chose to write C code, I'm often thinking of precisely how a computer will execute the high-level solution that I've sketched. Usually, given a problem, I am decomposing it into executable bits, like looping through a list, building a tree just-so, etc. I know what I want, and if I'm wrong then its my foot that I'm shooting.
C# -- "Rapid application development"
I'm only beginning to use C# regularly, but I'm finding it very useful for achieving simple tasks, particularly those that require a UI. .NET is providing an extensive collection of support libraries, so that loading graphics, interacting with them, etc is mostly a matter of connecting the pieces together. To me C# is called for when my problem is UI-centric: click the button and an image appears....
Perl -- "Text, files and strings"
My Perl use has been declining over the years, not for a dislike of the language, but rather a shortage of problems that call for its use. Perl is my tool of choice when I need to work with lots of text and files. I've used it to write everything from copy files between directions and rename to parse MBs of raw data and compute statistics for particular results of interest. (The later eventually was rewritten in C for speed and portability).
I'm always thinking that one of these days I'll try learning more about a functional language. However, I quickly fall into the trap of thinking procedurally. "I know you're on an x86, so just let me write my assembly and we can be done."
Subscribe to:
Posts (Atom)