Preskill, (2006) outlines several components as essential in designing an evaluation system. All components seem to logically support building evaluation capacity within an organization.
Leadership commitment is acknowledging the evaluation as important. When the leaders commit to the evaluation, the team intrinsically understands the importance and they want to participate in the process. This is more that the “rah-rah” speech about evaluation this is time on task, putting out the resources, committing the time, and allowing the evaluation to be conducted uninhibited. In other words, it is full support for the evaluation.
Evaluation vision and philosophy: much like having a vision and philosophy guides an organization or a program, it can also guide an evaluation system. A philosophy of how the organization views evaluation can set the tone for evaluation. The vision statement might express what the team values about evaluation. Synder, N. and Graves, M. (1994) offer, “Having committed to a vision, organizational members begin to participate in shaping it, fashioning it to reflect their own personal visions– pictures or images they have in their hearts and minds about their futures and their contributions to the organization.”
Evaluation Strategic Plan: is the road map to conducting the evaluation. It explains succinctly how the evaluation will be conducted, who will participate, how data will be collected and results disseminated. It offers a mechanism for updating and communicating throughout the process.
Evaluation Design and Implementation Requirements: This component speaks to creating an evaluation plan that is logical, flexible and is offers an open communication system. It ensures evaluation is conducted in context and that stakeholders have an opportunity to benefit from the findings.
Personnel and Financial Resources: This component I find directly linked to leadership commitment. If the leader commits to an evaluation system, adequate time, finances, resources are allocated. It is the insurance that evaluation will occur.
Reference:
Synder, N. and Graves, M., (1994). Leadership and vision-importance of goals and objectives in leadership. Business Horizons. Retrieved from: http://findarticles.com/p/articles/mi_m1038/is_n1_v37/ai_14922906/?tag=content;col1
Resources:
Preskill, H., and Catsambas, T.T., (2006). Reframing evaluation through appreciative inquiry. Thousand Oaks, CA:SAGE publications.
Posted in HRE585 February 24, 2010
I developed a survey tool; trying to write appreciative questions was the most dificult part of this assignment. I used the textbok and examples presented as a guide. I spent a lot of time trying to make the questions impartial and not just “happy” sounding or leading. It was important to try and elicit how someone felt about a particular aspect being queried.
Field testing an instrument is quite important. What you tend to do is read the survey the way you intended it to sound; getting another view not only helps you edit but helps you clarify questions. My classmate was able to offer insight about layout ..this is important in how the reader will percieve the tool.
I think field testing was the biggest take away for me; clarity in the questions, posing direct non-leading questions is essential to getting good results.
Posted in HRE585 February 16, 2010
My program model depicts a hypothetical nursing program at a community college. It is logical as a stand alone project but I am having difficulty seeing how to drill this down to one aspect of evaluation, namely the learning technology. I think from this stand point I would need to evaluate different areas of the program in order to create a comprehensive evaluation (though this is may be useful it seems extremely time consuming).
It is easy to see how AI can be used effectively to focus evaluation efforts especially when multiple stakeholders representing different interest are called together. As discovered last week AI could be quite useful in finding common ground and thus focusing the evaluation. I think as you plan the evaluation it is helpful to anticipate how you would use AI; this potentially guides data collection methods employed.
Program: HCC – Nursing Program Logic Model
Situation: State reports indicate a continued national shortage and immediate need for registered nurses in the geographic area of the college. The HCC nursing program intends to educate students to be competent, effective entry level nurses who will gain employment as RN’s and positively impact the world community.
Program Goal: The nursing program intends to educate students to become competent, effective, licensed Registered Nurses who serve area health care institutions. Graduates of the program will be perceived by employers as knowledgeable, professional and innovative. They will be able to address health care needs using available resources, technology, and research.
Resources: Area nursing programs, state board of nursing, state community college board, nursing association, student nurse association, teaching materials, community partnerships, technological equipment
(click image to enlarge)
Posted in HRE585 February 9, 2010
People are familiar with traditional evaluation processes, they seem to expect them and accept how they are delivered. Members of organizations have been conditioned to expect quantitative data results will be reviewed with them upon completion of an evaluation for determination of what area needs to be fixed. Since AI is a participatory process, stakeholders may have a difficult time understanding the philosophy and becoming engaged in the evaluation. Some participants may question the validity of the process while others may refuse to participate. Holmberg, K. (2009), reports resistance to engaging may stem from anxiety or fear that the process will not yield quantifiable results. I think it would be helpful if someone from the organization/institution had some idea of AI prior the evaluation. Without this background knowledge it may be difficult to get the process off the ground.
Bush, (2007), describes a situation where there was no useful data (no good inspiring stories) that came from the group during the discovery phase. Though this may have been related to participants being unwilling to share, Bush suggested these participants had no story of value to add to the process. He further described the dreadfulness of this nuance on him as an evaluator and detailed how it stalled progress.
I think the biggest challenge in using AI would be getting participants to understand this approach as legitimate.
References:
Bush, G., (2007). Appreciative inquiry, it’s not (just) about the positive. OD Practitioner, v39(4), p. 30-35. Retrieved from: http://www.gervasebushe.ca/AI_pos.pdf
Coghlan,A., Prenskill, H., Catsambas, T., (2003). An overview of appreciative inquiry in evaluation. New Directions in Evaluation, issu 100, p.5-22. retrieved from:http://blogs.ubc.ca/evaluation/files/2009/02/appreciative20inquiry.pdf
Holmberg, L., and Reed, J., (Aug/2009). (eds) AI Practitioner, v11(3). Retrieved from: http://appreciativeinquiry.case.edu/uploads/AIPAug09ResearchNotes.pdf.
Posted in HRE585 February 2, 2010
I initiated a Google search of appreciative inquiry and looked for specifics on evaluating learning technologies; I found the following resources helpful and appropriate to share:
Appreciative Inquiry Commons, is a website dedicated to appreciative inquiry. The site is replete with tools for practice, sharing boards, and research in AI. Articles detail use of AI across different sectors; of note is the annotation for “Emerging technologies in global communication: Using appreciative inquiry to improve the preparation of school administrators,” (authors: Raymond Calabrese , Brian Roberts , Scott McCleod , Rae Niles , Miles Berry, 2008); and Donovan, L. Shirley, G. and Derby, S.,(May 2008). Wired discovery: New conversations and deeper connections. AI Practitioner, The International Journal of AI Best Practice. Retrieved from: http://www.aipractitioner.com/downloadable/download/sample/sample_id/21/. The journal requires purchase but the sample proves inviting.
This volume of AI Practitioner posts an interview with Agneta Nilsson as she prepares to engage in a software improvement process. In this short interview, Nilsson offers her depiction of the value of the AI process related to information communication technology. The reader is also encouraged to read the other research notation in the research notes section as it serves to offer perspective on engaging the AI process.
Holberg, L, and Reed, J. ed.,(2009), Research notes. AI Practitioner, vol 11(4). Retrieved from: http://appreciativeinquiry.case.edu/uploads/AIPNov09researchnotes.pdf
This PowerPoint presentation was presented by Prenskell at the 2007 AEA (American Evaluation Association) CDC (Centers for Disease Control) summer evaluation institute. The slides depict the steps of AI as an evaluative process and offer a workshop format complete with tools to practice the process. This resource can be helpful in adding clarity to the how the process ensues and offering details of how to frame questions. It is succinct and I found it gave me additional perspective to the AI process.
Prenskill, H. (2007), Using appreciative inquiry in evaluation practice. AEA/CDC Summer Evaluation Institute. Retrieved from: http://www.eval.org/summerinstitute07/handouts/si07.preskillf.pdf
References:
Appreciative Inquiry Commons website: http://appreciativeinquiry.case.edu/news/eventDetail.cfm
Calabrese, R., Roberts, B.,McCleod, S., Niles, R., and Berry, M., (2008). Emerging technologies in global communication: Using appreciative inquiry to improve the preparation of school administrators. AI Practitioner,The International Journal of AI Best Practice.
Donovan, L., Shirley, G., and Derby, S., (May, 2008). Wired discovery: New conversations and deeper connections. AI Practitioner, The International Journal of AI Best Practice. Retrieved from: http://www.aipractitioner.com/downloadable/download/sample/sample_id/21/
Holberg, L, and Reed, J. ed.,(2009), Research notes. AI Practitioner, vol 11(4). Retrieved from: http://appreciativeinquiry.case.edu/uploads/AIPNov09researchnotes.pdf
Prenskill, H. (2007), Using appreciative inquiry in evaluation practice. AEA/CDC Summer Evaluation Institute. Retrieved from: http://www.eval.org/summerinstitute07/handouts/si07.preskillf.pdf
Reflection: There were several different principles that my colleagues highlighted discussing the eight core principles; all 8 principles were not discussed but of the ones selected there was some common ground. We seem to agree that regardless of the principle, AI offers an opportunity to bring people into the process for a better outcome. The constructivist principle fosters creating what you will focus on (determining own reality); free choice principle supports increasing participation by allowing participants to determine where they will spend their energy; the wholeness principle recognizes the value in bringing all stakeholders to the table, anticipatory principle prompts participants to look forward and the positivity principle keeps participants motivated and enthusiastic. I also see the principles without clear delineation; they seem to operate off each other to create the framework for operation.
Posted in HRE585 January 26, 2010
What I think I enjoyed most about the discussion this week was seeing ethical dilemmas from different evaluation standards; though proprietary standards specifically address ethical concerns, it is imperative evaluators recognize ethical concerns throughout the process. I also noticed that altough solutions were similiar, most people had difficulty selecting only one standard they felt was being violated. Ethical issues ranged from context presented in the case studies (evaluation team with little expertise, manager not following protocol, ) to projections about how the findings might be interpreted (someone offered an example of what might happen with skewed results presented to a school board). Further, I appreciate that situations need to be reviewed for context and impact before evaluations occur.
Posted in HRE585 January 20, 2010
My name is Sheila and I must admit I am learning a great deal in this program. I am the director of nursing at a community college in Chicago. I have been faculty for 9 years and certainly enjoy education! My undergraduate degree is from Bradley U. (go BU) and my MSN is from Governors State University. My husband and I have a daughter in her second year of college, and two sons, one a senior in high school and the other a freshman. I’ve grown to like the online format for classes; it allows me to work late at night when the house (including the dog) is asleep.
Posted in HRE585 January 8, 2010
I enjoyed this course on several levels; 1) it was a change from all of the technicals of technology and 2) it allowed me to shift from how to do into why we do mode.
I must admit I learned something about a number of topics that I did not know. I now understand digital media material does not require formal copyright to be entitled to copyright protection. I do not think I gave much thought to information posted on the internet as a source to be protected, I have a much greater understanding of the process and although I am not convinced that freeware is the Utopia for information sharing, I do not believe, everything needs to be proprietary.
Most interesting, I think was the discussion of anonymity on the internet; this offered me a chance to reconsider my perspective. I narrowly focused on the right to disclose/expose identities of those who post anonymously on the internet. As a believer in accountability, I did not see value in anonymous post. I resigned that being anonymous was equivalent to being coward; and if you believed in what you spoke, you should proudly attach your name. I still believe anonymity provides unnecessary cover for those who seek to hurt others, like cyber stalkers; “…where a potential stalker may be unwilling or unable to confront a victim in person or on the telephone, he or she may have little hesitation sending harassing or threatening electronic communications,” Ascroft, J. (2001). I do realize however, it is imperative that we make some exceptions. Whistleblowers for example, may require the protection from retaliation that anonymity provides; this seemingly simply concept has broadened my view.
Great course..thanks.
References:
Ascroft, J. (2001). Stalking and domestic violence report to congress. U.S. Department of Justice, Office of Justice Programs. Retrieved from: http://www.ncjrs.gov/pdffiles1/ojp/186157.pdf .
Resources:
Fisher, W., (11/28/09). Politics –US: High court to hear Patriot Act challenge, Global Issues, Retrieved from: http://www.globalissues.org/news/2009/11/28/3696
National conference of state legislators, report http://www.ncsl.org/default.aspx?tabid=13495
Posted in EPS415 December 28, 2009
Until there is an application that ensures equal access and by that I mean not only physical access to computers and the internet but also technology literacy access, then we will have two groups of learners. The two groups however, will be distinguished not by their ability to use specific devices but how narrowly or broadly they use the technology. For example, while many low income families might have cell phones, the primary uses are for emergencies and socialization. Compare this to an individual who is indoctrinated to use a cell phone as a tool to check stock market results, or collaborate on learning projects. The aforementioned is clearly at a disadvantage. The digital divide demonstrates a gap between what to do with technology; how to exploit it for greater goods verses just being ripe consumers.
It is also interesting to note that the use of the internet itself dictates a change in the way we teach driven by a new way of learning. A government summary report from the web-based education commission, 2001 notes:
The need for a new design in learning is there. Today’s education is built on an agrarian model that worked in the years when we were a nation of farmers, foresters, and fishermen. Schooling changed to take on elements of the industrial revolution (factory-line classes, assembly-line curriculum, and teacher-foremen) that worked for the needs of the Industrial Age. New designs are needed to create the “knowledge workers” who will define the Information Age.
Part of the problem identified from the above quote, is that we are thinking in our brick and mortar fashion and trying to adapt technological changes to old code. Think copyright problems with digital media; anticipating the next wave may have us as John Newton “prognosticates” developing a more simple and expedient language, using body gestures to control communication devices, and incorporating high level voice recognition to provide service. Everyone will keep a certain stride with technology advancements. Those who learn it as an inherent system that connects all things will probably be more intuitive.
It is possible that we continue to add new technologies at such a speed that unless we create information forward thinkers who anticipate technology we will indeed have two classes of people; those who use technology as it is given and those who are defined by conventional limits of technology. There will be no option for people not manipulate technology; the question is will responders outpace innovators?
Reference:
Newton, J. (2007, March, 14). Information technology 2010. Message posted to http://newton.typepad.com/content/2007/03/information_tec.html
The Power of the Internet for Learning: Final Report of Web-Based Education Commission (2001). Retrieved from: http://www.ed.gov/offices/AC/WBEC/FinalReport/index.html
Resource:
Sacco, A. (2007). Giving cell phones to low-income households could create 11B in jobs. CIO. Retrieved from: http://www.cio.com/article/207450/Giving_Cell_Phones_to_Low_Income_Households_Could_Create_11B_in_Jobs_Study_Finds
Posted in EPS415 December 21, 2009
So I created a Facebook account as an “investigation.” I’ve not been on Facebook nor had I desired to try. I asked my 14 year old for assistance, he was so excited for me he spelled my name wrong. As he walked me through, growing impatient with my reading each question in the profile I thought for a fleeting second that I might leave it up a few days and see how it works. I was nervous about entering my personal data, email address, date of birth and the like. I don’t usually post pictures to the internet and knew for sure I would not want to do this now. What I found most interesting was the reaction in the kitchen where I worked on the laptop. Once my profile was complete, I was able to search for “friends.” I chose something I thought pretty benign and searched for the people in the house; my niece, daughter and husband came up immediately. There was excitement in the kitchen now, me with a brand new Facebook page (and receiving directions from Facebook officials), “check out Dad’s page” “Find Angie, add her as a friend,” “you know you can write on their wall.” I saw my cousins, my sister, nephew, yet I was not compelled to stay connected. I can see how people might get caught in the frenzy however. You can get quick updates on people you have not seen for years, reconnect with friends, meet new people, set up appointments; query like minds, the list is endless. It appears to be an online version of playing catch up or what’s up? It makes me wonder how connected do people really feel? My niece just put her Facebook page up, she has 181 “friends”, my daughter has been on a year, and she has over 1,000 “friends.”
Incidently noted was the press by Facebook to keep you acquainted. When I went to disable my facebook page, I was queried as to why I wanted to make such a move. Even more interesting, was every choice you can select (except – I’m leaving because I already have an account) offers you assistance to manage the problem. For example, if I selected I was leaving because I did not find Facebook useful, a pop up explaining various uses attempted to dissuade me.
Perhaps this type of socializing is the limit of how we really want to stay connected anyway. I have learned over the years that I rarely miss people who I am no longer connected to – in the real face to face connected way. When we see people who we have not spoken to in years, how long do we really stay “reconnected?” Social networking technology allows us to keep up with the tidbits and milestones without forcing us to have relationships.
I also wondered how effective this could be as an educational tool. According to Science Daily, when students are engaged on Facebook (or similar) social networking tools, “they are practicing the kinds of 21st century skills we want them to develop to be successful today.” Included in that list are “positive attitudes towards technology” and finite computer skills. The trick perhaps, is to determine how to effectively use Facebook as an educational supplement.
Do shy people have a better time talking on social media? Does this media mimic face to face connections? If students are actually learning practical, technical skills are they also learning social, communication skills?
Reference:
Educational benefits of social networking uncovered, (2008, July, 21). Daily. Retrieved from: http://www.sciencedaily.com/releases/2008/06/080620133907.htm
Resource:
Facebook home page. Retrieved from: http://www.facebook.com/
Posted in EPS415 December 14, 2009
Previous Posts
