Net url: http://eklavyasai.blogspot.in/2012/03/improve-practice-of-software.html
Last Updated On 26th January 2013
Given below are comments I received from Dr. Mark Tarver for my academic paper, "Improve the Practice of Software Development in India by Having a Software Development Career Track in Indian CS & IT Academia" whose preprint version is available on arxiv.org hosted by Cornell University, USA: Abstract; Pdf.
March 23rd, 2012
Dr. Mark Tarver's Comments on an earlier version of the paper,Abstract,PDF
Dr. Mark Tarver of lambdassociates.org, who is also a former UK CS academic, was kind enough to share his valuable comments on the above paper. He further kindly permitted me to post his comments (and my response) suitably edited for a blog post, here. As I do not have express permission to share his comments using CC-BY license, his comments are excluded from the general CC-BY license applied to this blog post/page.
Let me take this opportunity to thank Dr. Tarver for his very frank and very well expressed views in his articles on CS academia. These articles and some of Prof. Bjarne Stroustrup's articles and a few private mail exchanges that an internationally renowned software development practitioner and CS academic kindly had with me, gave me the courage to challenge Indian CS/IT academic mind-set regarding software development practice.
Of course, my views may have flaws. Prof. Stroustrup and Dr. Tarver may disagree with part or all of my views. But, in my considered opinion, the situation is so dire in Indian CS/IT academia, in general, that it has almost become dysfunctional! There may be some exceptions, of course, but, in general, in my considered opinion, CS/IT students are getting a very raw deal. CS/IT academia in other countries may also be partly dysfunctional. We urgently need discussion followed by concerted attempts of corrective action to improve life for CS/IT students & employers.
Dr. Mark Tarver (MT) wrote: I read your paper with interest; and I agree with the thrust of what you say.
Ravi S. Iyer (RI) responded: That is vital input for me. Thanks again.
MT: Two points to consider.
1. Industry is rarely innovative and trend-setting in terms of practice and computer-language development. Practices linger on in industry due to the costs of adoption long after they are left behind by best academic practice.
RI: I tend to agree with your views here, though I do not have wide spread exposure to world wide academia to comment on "best academic practice".
MT: Lisp, Prolog, ML were all developed at university. It was industry, not academia, that kept Cobol alive for a long time.
RI: Industry marches to the beat of a different drummer, the money making drummer :). I agree.
MT: Hence placing the criteria for academic promotion in the hands of commerce is likely to stultify innovation in software practice.
RI: Very interesting point. In India, software practice in academia is quite poor. While we do have an Open Source community in India, as far as I know, it is nowhere near the thriving Open Source community that say, the US, has. So, I felt that industry should get involved in the "peer review" in the initial years.
Over time, if Indian CS/IT academia itself acquires the capability to "peer review" software development contributions, then industry can be dropped from the "peer review" process. But, as of now, in India, industry is the only community, I think, that can deliver on software development "peer review" for the huge numbers of students that India has.
MT: I would say it needs to be put in the hands of the open source community.
RI: I think, this would be a wonderful option for countries like the US and UK. And, if it proves itself as a sustainable and successful process, then other countries like India can follow their lead.
MT: 2. There is a third class of teacher who is neither productive in terms of writing (commercially significant) code or in writing papers but who is simply an excellent teacher. This sort of person is totally under-served at the moment.
RI: I entirely agree. In fact, I have at first hand seen how teaching excellence is neglected. You may want to read my blog post, CS & IT Academia: Is Teaching Excellence Important?, where I conclude by writing, "I think I am an old school guy who feels that the FIRST and FOREMOST DUTY of a TEACHER is to TEACH and TEACH WELL."
Tuesday, June 4, 2013
Concrete Suggestions for Measuring Teaching Quality in Practice-Oriented Computer Science/Information Technology streams
Net url: http://eklavyasai.blogspot.in/2013/06/concrete-suggestions-for-measuring.html
Last updated on 6th June 2013
At the outset, I request the kind indulgence of readers for blowing my own bugle a little, with the intention of substantiating my case.
Yesterday I received an email from a recently passed out M.Tech. (Computer Science) student (I don't know if exam results have been declared yet but the student is a high-flyer and will, in all probability, pass). He thanked me for my C++ classes which he felt played a significant role in him clearing a job interview and getting a job offer which he intends to take up.
I must thank you for getting this offer because my interview was solely based on C++ programming and very little of algorithms. Even the latter was with respect to data structures that are part of the C++ STL. Your C++ classes, and at a later stage your slides were of great help to understand the basics of C++. Therefore, I felt this urge within to inform you about this.
--- End passed out student message extract ----
I would have taught him the C++ course when he was in Ist M.Sc. (Maths) in academic year 2009-10. The department is a Mathematics & Computer Science department. Many of the two year M.Sc. (Maths) program students continue on to do the two year M.Tech. (Computer Science) program. [BTW in March 2012 I parted ways with the above-mentioned department and educational institution.]
Now I would not like to comment much on why the tech. interview (as per the student's description) did not cover algorithms in some depth. IMHO, ideally there should have been a balance of questions on both algorithms and the particular programming language skills they are interested in, which was C++ in this case.
However, the reality of the situation is that the practical programming skills imparted to the student via the lab. course was instrumental in him clearing the job interview and landing the job. So the teacher concerned would seem to have done a good job and would earn the appreciation of academic administrators.
But the way things are run in UGC/AICTE regulated educational institutions in India, with accreditation agencies like NAAC being powerful forces, good teaching quality like in the above case, are not recorded in any fashion in their evaluation metrics. If I am not mistaken, the teaching quality assessment according to UGC/ACITE norms is supposed to be or can be a self-assessment! Naturally all teachers will be assessed as good or excellent teachers then :-) (unless there are many theory exam failures).
The key measure that is recorded is the research publication record. NAAC (the key national accreditation agency for UGC institutions, I believe) seems to be focused mainly on the research publication record of institutions it accredits and grades. So academic administrators have become obsessed with research publication record to ensure that the institutions they administer get a good NAAC rating. This results in a scenario where academic administrators put immense pressure on teachers to publish research papers, and almost ignore promoting excellence in teaching quality.
Now, in my special case, I had retired from commercial work in Sept. 2002, and was offering honorary service of teaching lab. courses in a 'deemed' university. I was utterly disinterested in an academic career or in academic research (my designation during my stint was Honorary Staff, Honorary Faculty and finally Visiting Faculty). I was only interested in my spiritual career and part of that involved serving students by teaching them what they needed to know using my knowledge base of 18 years international software industry practitioner experience and also studying new knowledge areas required to teach specific lab. courses allotted to me. So my focus was on excellence in teaching lab. courses. I have had the satisfaction of teaching them to the best of my ability, given the circumstances, and receiving the love and gratitude of the students in return. From a spiritual and ex-software industry practitioner/professional career point of view I consider it to be a very successful stint (9 years) of spiritual and industry-quality service.
But from an Indian academic system point of view my question is, will the typical Indian CS/IT academic focus on excellence on teaching lab. courses like the C++ course I taught? IMHO, the clear answer is No (there may be some exceptions, of course - I am talking about the typical Indian CS/IT academic). As his/her bosses are mainly bothered about NAAC grading and UGC/AICTE norms for which the only thing that really matters is research publication record.
Further, lab. courses are looked down upon as unimportant courses which can be handled by junior staff like Teaching Assistants or Research Assistants (Research Scholars). The evaluation of lab. courses are very lenient. Rarely is a student failed and many students get the top grade. Theory courses have paper examinations with external evaluation. That is considered far more important in the (UGC/AICTE) Indian academic system. Theory courses have more credit than lab. courses. [If UGC/ACITE publish statistics about failures in CS/IT lab. courses in the institutions they regulate, I am quite sure it will be a miniscule number. However, the failures in CS/IT theory courses will be, I am quite sure, a not insignificant number, in comparison.]
This is very strange for a very practice oriented field like Computer Science/Information Technology. As the above freshly passed out student interview example clearly shows, the vital skill that is assessed during typical CS/IT job interviews is the practical skill. Of course, theory is important too and ideally there should be a balance between theory and practice. But, IMHO, an administration policy where the teaching of the practice of software development (lab. courses) is relegated to junior teachers and considered unimportant, is a disastrous education policy for a practice-oriented field like Computer Science/Information Technology.
What can we do to rectify the situation? Teaching Quality must be measured even if the measures may have some flaws in it (like the flaws in measuring research contributions). Over time the flaws in measuring teaching quality could be addressed/controlled. I have a few concrete suggestions for measuring teaching quality of lab. courses in Indian (UGC/AICTE) CS/IT academia:
1) Teachers who create course material for a course (as against using course material from other sources) should have their course material reviewed by other teachers just like research papers are reviewed. Appropriate credit should be given to teachers whose course material gets a good review. The review may include student assignment submissions including source code, so that reviewers get an idea of how students are benefiting from the course.
2) Lab. courses should be evaluated as strictly as theory courses. It should have external examiners like theory courses. There should be no hesitation in failing students who fare poorly in assignment submissions and exams. Administrators should examine lab. course result patterns and compare it with theory course result patterns. Very high success rates in lab. courses contrasted by quite different success rates in theory courses should ring alarm bells and invite investigation. If the teachers concerned are found to have done an overly lenient job of evaluation they should be penalized.
Additional suggestions to improve the teaching of the practice of software development, not related to measuring teaching quality, are:
a) Lab. course credits should be at least equal to theory course credits. Further, the number of lab. courses should be roughly equal to the number of theory courses. [I believe, the typical current ratio is 4 to 5 theory courses and 2 lab. courses in a semester, with theory courses having 3 credits and lab. courses having 2 credits.]
b) The practice of relegating lab. courses to junior teachers must be abandoned. Given the vital importance of practice in CS/IT, senior teachers should teach lab. courses.
Initially I had suggested student feedback as one measure of teaching quality, as follows:
*) Student feedback about the course should be collected. Industry training institutes heavily use participant feedback to get an idea of the quality of teaching and suggestions for improvement. Academia can learn from them. Yes, there may be some serious concerns like students using it as a weapon against the teacher, and the teacher targeting classes where he/she receives bad feedback. But there surely must be ways to mitigate such issues. Students are the key customers/stakeholders, usually paying customers, of the education system. Teachers in academia must learn to respect this fact and learn to accept criticism from students about their teaching, just like teachers in industry training institutes respect (or are forced to respect) criticism from participants/students of their courses.
--- end suggestion ---
I received vehement criticism from a US CS academic (over two separate mails which he allowed me to share with others). I have shared below the key parts of the mail exchange:
The US CS academic wrote:
It may not have occurred to you but some students punish good teachers for being demanding and for not being soft at grading. I have seen the most outrageous *anonymous* student feedback for a professor who caught many cheating. The easiest way for a professor to get a good rating from anonymous feedback is to lower standards and give higher grades.
My response was: Oh Lord! While I thought that certainly there were possibilities of students using the anonymous feedback as a weapon against the teacher, I did not know that they actually do go that far. You have the experience of such feedback systems in academia - I don't (though I do have industry feedback experience, which I think is a very different ball game). I am quite shocked to know of this. Hmm.
The US CS academic wrote back:
Note that anonymous accusations traditionally are absolutely disregarded or in cases even illegal under Western law. Except, it seems for professors. Disgusting.
My response (slightly edited to give more clarity) was: Thank you so much for bringing out this aspect so clearly. I think an academic who gets targeted via such anonymous feedback/accusations would go through horrifying emotional torture. He/She would feel there has been no justice, no chance for him/her to prove his/her case. Hmm.
I really need to pause, think hard, and ensure that my blog post does reflect this horrifying abuse possibility. And then explore whether there is a "safe" way to use student-customer feedback as one measure of teaching quality. As of now, I am quite clueless about suggestions on how to prevent abuse of student feedback like the cases you mentioned and how to address/prevent/resolve the 'disgusting' nature of anonymous feedback accusations being considered as a negative measure of teaching quality.
After learning about the US academic's experience with abuse of anonymous student feedback to punish good teachers I realize that my earlier view that "there surely must be ways to mitigate such issues" may be wrong. I am not so sure any more about ways to prevent or even mitigate such issues in the Indian UGC/AICTE regulated institutions context. [For the 'cheating' example shared above one could ignore feedback from such students but they could influence their student-colleague-friends to give bad feedback.] My experience of teaching in academia is limited to an educational institution with a focus on human and spiritual values where the students were exceptionally well behaved and had tremendous respect for their teachers. Perhaps there is a very real possibility of such abuse of anonymous student feedback to punish good teachers in some other Indian educational institutions.
Direct (non-anonymous) student feedback used as a measure of teaching quality, when negative, has the possibility of seriously damaging the human relationship between student and teacher. So I don't think that can be considered.
Therefore I withdraw the suggestion of using student feedback as a measure of teaching quality.
Ravi S. Iyer, June 6, 2013 at 9:04 PM
Shankar Balasubramanian wrote over email:
It is definitely true that using student feedback to measure teaching quality of an individual could be biased due to (mis) understanding of students, who may be a bit immature in judging the quality of teaching recd.
Its also determined by peer influence or just to have fun as a prank in belittling the teacher without realising gravity of their deed.
Our Scriptures treat Guru / Teacher as GOD, Matru Devo Bhava, Pitru Devo Bhava, Acharya Devo Bhava and all divine personalities have always shown to common man that by revering their Guru, the students achieve success in all aspects of their life.
But feedback when given in a matured way can give assessment of quality of teaching of an institution. Its like a 360 degree appraisal in Management Parlance. It will also be an indicator to the teacher on what the students expect when they receive the lessons.
So maturity on part of both teacher and the taught will help in assessing the quality through a good feedback.
Ravi S. Iyer, June 6, 2013 at 9:15 PM
Thanks for the comment, Shankar. I think your view has some valid points, for sure. But if abuse cannot be prevented then using it for assessing teaching quality of teachers is very risky.
However, using the feedback informally is a different matter. I believe teachers in UGC/AICTE universities are free to do so. But it will not reflect in their appraisal. So only interested teachers will go that far - BTW I regularly collected feedback from my lab. course students in an informal fashion, and that feedback did help me in ways similar to some of the points you mentioned in your comments.