Method
To implement and evaluate our instructional project our group decided to use the small group implementation model. None of us currently have students, therefore we decided the best dissemination method would be to compose an email with instructions, our instructional webpage, and a Google Form for feedback. For the instructions we asked the students to work through the modules at home and at their own pace, complete one activity, and submit feedback. The Google Form contained demographic questions, site usability and navigation questions, their overall impression of the site, and suggestions for improvement. We also asked which activity they completed, the length of time it took, and what they learned.
To find students that would be willing to work through our instructional webpage, we decided to ask friends and family. In total we were able to get feedback from 10 students. The small group implementation method allows for the course designers to conduct usability testing with similar learners. The course was designed for high school seniors, however our student population contained 4 freshman, 1 sophomore, 2 juniors, and 3 seniors. Our students where evenly split between the genders, with 6 female students and 4 male students. We do realize that because our student pool was drawn from our own family members and our friends’ children the participants socioeconomic background and views on education maybe different from our target population.
Results
The feedback we received from our high school students were mostly positive. Most of the students completed at least one activity and from our feedback, they each tried out a different activity in the modules. This makes us think that each of the modules and activities works well and the students who tested them are satisfied. There were a few suggestions on what did not work, although they did not give specifics on which module they had issues with. A few examples were: they had a difficult time uploading part of one of the modules, the modules opened too many tabs and it was difficult to go back and forth through them all, to change one of the example pictures, and finally one of the links did not work and they could not move forward. When we asked what was at least one fact you learned they answered in a way that we felt they understood the objective of interpersonal skills and making a good impression in the business world. They understood that the way you need to conduct yourself in the business world even as a high school student is important. They noted a person is viewed through their appearance, manners, attitude, and in the way they communicate through all platforms. Overall, they liked the look, the ease of use of our website, and in our opinion, they understood the objectives we created for this website.
Refection
From the comments collected in our survey, overall, the students had a positive perception of our site. While our usability sample was small and not balanced over grades and gender, we did receive useful feedback which can be incorporated into improving the site:
We learned that students may not be familiar with such terms as usability and curriculum, therefore, our questions could be strengthened by adding more colloquial language that students understand. Another weakness in our questionnaire was not providing an option list for the tools not working, so we are left to guess which was not working well based on the student’s description. For example, one respondent reported on usability by thoughtfully responding, “For some of the activities there were too many tabs to open and go back and forth to.” This student reported in an earlier question they were working in “The appropriate work attire” activity, so we can infer it was Module 1 they were working in and therefore we can use the feedback to help make improvements. However, it would have been more useful if the feedback could be tied specifically to the exact area of the site where they encountered the difficulty. Additionally, when respondents were asked about finishing the instruction, we did not direct them to specific instruction to complete for this test, so the feedback was general. All students checked true that the site lined up with the instructional goals, and it was easy to learn from the site, however, they didn’t actually complete the training end to end, so end to end usability testing would improve our results on this question.
Here is the specific feedback that led to the above conclusions on improvement:
Question: From the homepage was it easy to determine where to begin? All students surveyed reported it was easy to determine where to begin from the homepage.
Question: From module 1, was it easy to get to module 2? Only one female could not easily navigate between module one and module two.
Question: Was anything on the website not visible or working? From the responses we learned “the writing tool” was not working for all students. Since the student did not specify the module, we can only assume it might have been the Padlet in Module 1 or the Scribed email in Module 2, or any of the survey tools in Module 3. The takeaway is that all functionality should be tested in all browsers for compatibility and working access. Directions should also be strengthened for the use of these tools, for example, if an account is needed to be created, and if there is something specific the student needs to do to make sure the tool is accessible to them.
Question: Was the overall site easy to use and navigate? All respondents felt the site was easy to navigate.
Question: How long did it take you to finish the instruction (minus completing the activities)? Instruction took between 5 and 30 minutes and was not module specific.
Question: The Instruction on the website was effective in helping me complete activities.
All agreed instruction was effective in helping to complete activities.
Question: It was easy to learn with this website. All checked TRUE that it was easy to learn from the website.
Question: I feel like each module lined up with the instructional goals. All checked TRUE that each module successfully lined up with the instructional goals.
Question: I like the way the information on this website was arranged. All checked TRUE I like the way the information on this website was arranged.
Question: Specify which activity or activities did you completed.
The respondents chose different activities. One reported the Appropriate Work Attire did not work.
Question: Is there anything on the website that does not make sense? If so what?
One respondent was unable to upload their work as instructed in the module site. All others reported positively that the website made sense. One did not answer.
Question: How would you improve this website's usability?
Some respondents did not understand the word usability and this was clear because they responded that all students who need to improve interpersonal soft skills should use the website. This is not what we were asking about with the use of term usability. Two respondents were unsure and replied yes - giving no detail how they would change usability and one did not respond. One reported on usability thoughtfully by responding, “For some of the activities there were too many tabs to open and go back and forth to.” It would have strengthened our feedback to know specifically which module this defect was discovered in.
Question: What changes would you make to the curriculum?
When asked how they would change the curriculum we received the following suggestions: “For the pink hair dress code one, don’t use that picture use something with out of order clothes.” Another responded “I like the curriculum but I liked module 3 best because it had a lot of very good information and resources adding, “Maybe just add more content.” One student responded N/A, suggesting they didn’t think the word ‘curriculum’ applied to them. This may mean we should have asked the question using more colloquial language.
Question: What specifically do you like about this website?
In response to specifically what you like about this website we received the following responses: “i like how you can complete all of these activities to help students know how to be professional in a work force”; “It asked questions and made me think.”; “the videos”; ”Every thing”; ”I like module 3, I found the videos really interesting”; “I enjoyed looking at the content” and ”It was easy to follow and understand.”
Question: What specifically did you not like about this website? In response to What specifically did you not like about this website we received the following input: there wasn’t anything i disliked about the website, How does of the links didn’t work, so i could not move forward, I didn’t like that some modules had more content than others, it felt unbalanced. I didn’t like that some modules had more content than others, it felt unbalanced. That I couldn’t use the notes website thing, Nothing and one N/A.
Question: What is at least one fact or idea that you learned from this website about interpersonal skills and making a good impression in the business world?
In response to What is at least one fact or idea that you learned from this website about interpersonal skills and making a good impression in the business world? We received the following: appearance is an important factor when going into the workforce. it can give people perspective on how you act and your personality, Dress appropriately and use manners, and a good attitude., email etiquette, To be nice,That being able to read people in the business world is key to being successful and professional, That you should have a good work attire or people will think that you won’t be able to perform your job very well, That you should have a good work attire or people will think that you won’t be able to perform your job very well. Not to use exclamation marks in a professional email
Question: Overall, I enjoyed this website. All agreed Overall, I enjoyed this website.
From a collaboration standpoint, working as a team to develop the site, we struggled with a site template and consistency in our modules, which some end users noted. The lesson learned here is to develop a site template which details how each module will be constructed very specifically, including the tools from weebly, like the survey, that will be utilized in building the module and then stick with it, even if there are multiple content authors. As a team we could have been more consistent in the way we developed the site across all three modules. Another aspect of this assignment that it was difficult to navigate, as a team, exactly what multimedia artifacts needed to be created and how they should be designed overall in the context of the site. Each module author developed their own content and delivery, but taken as an overall site, it is clear that we, as module authors, had individual visions, and it is not clear if overall the vision matched what we delivered. It is also important to note that the original blueprint of the instruction was a more brick and mortar approach to instruction, and this plan had to be modified for delivery in a website, multimedia platform. It may have made more sense to have had one team member serve as the site developer. In the current limited summer time constraint of this course, that was not a viable option.
In conclusion, our site received positive suggestions and good feedback that we can use to make it even better. It was a good experience to approach the development of a site from the ADDIE instructional design model and to work together to incorporate into our design useful content to improve the interpersonal skills of high school seniors.
To implement and evaluate our instructional project our group decided to use the small group implementation model. None of us currently have students, therefore we decided the best dissemination method would be to compose an email with instructions, our instructional webpage, and a Google Form for feedback. For the instructions we asked the students to work through the modules at home and at their own pace, complete one activity, and submit feedback. The Google Form contained demographic questions, site usability and navigation questions, their overall impression of the site, and suggestions for improvement. We also asked which activity they completed, the length of time it took, and what they learned.
To find students that would be willing to work through our instructional webpage, we decided to ask friends and family. In total we were able to get feedback from 10 students. The small group implementation method allows for the course designers to conduct usability testing with similar learners. The course was designed for high school seniors, however our student population contained 4 freshman, 1 sophomore, 2 juniors, and 3 seniors. Our students where evenly split between the genders, with 6 female students and 4 male students. We do realize that because our student pool was drawn from our own family members and our friends’ children the participants socioeconomic background and views on education maybe different from our target population.
Results
The feedback we received from our high school students were mostly positive. Most of the students completed at least one activity and from our feedback, they each tried out a different activity in the modules. This makes us think that each of the modules and activities works well and the students who tested them are satisfied. There were a few suggestions on what did not work, although they did not give specifics on which module they had issues with. A few examples were: they had a difficult time uploading part of one of the modules, the modules opened too many tabs and it was difficult to go back and forth through them all, to change one of the example pictures, and finally one of the links did not work and they could not move forward. When we asked what was at least one fact you learned they answered in a way that we felt they understood the objective of interpersonal skills and making a good impression in the business world. They understood that the way you need to conduct yourself in the business world even as a high school student is important. They noted a person is viewed through their appearance, manners, attitude, and in the way they communicate through all platforms. Overall, they liked the look, the ease of use of our website, and in our opinion, they understood the objectives we created for this website.
Refection
From the comments collected in our survey, overall, the students had a positive perception of our site. While our usability sample was small and not balanced over grades and gender, we did receive useful feedback which can be incorporated into improving the site:
- Testing browser compatibility for the entire site with multiple browsers to make sure all features function for all users: Safari, Firefox, Chrome, Internet Explorer and Edge.
- Consider the users suggestion that out of order clothes be used for the “pink hair dress code” example.
- Consider the balance of content across modules and discern if more content or instruction needs to be added.
- Improve the usability testing questions and add end to end testing when delivering this module in the real-world scenario for which it was designed.
We learned that students may not be familiar with such terms as usability and curriculum, therefore, our questions could be strengthened by adding more colloquial language that students understand. Another weakness in our questionnaire was not providing an option list for the tools not working, so we are left to guess which was not working well based on the student’s description. For example, one respondent reported on usability by thoughtfully responding, “For some of the activities there were too many tabs to open and go back and forth to.” This student reported in an earlier question they were working in “The appropriate work attire” activity, so we can infer it was Module 1 they were working in and therefore we can use the feedback to help make improvements. However, it would have been more useful if the feedback could be tied specifically to the exact area of the site where they encountered the difficulty. Additionally, when respondents were asked about finishing the instruction, we did not direct them to specific instruction to complete for this test, so the feedback was general. All students checked true that the site lined up with the instructional goals, and it was easy to learn from the site, however, they didn’t actually complete the training end to end, so end to end usability testing would improve our results on this question.
Here is the specific feedback that led to the above conclusions on improvement:
Question: From the homepage was it easy to determine where to begin? All students surveyed reported it was easy to determine where to begin from the homepage.
Question: From module 1, was it easy to get to module 2? Only one female could not easily navigate between module one and module two.
Question: Was anything on the website not visible or working? From the responses we learned “the writing tool” was not working for all students. Since the student did not specify the module, we can only assume it might have been the Padlet in Module 1 or the Scribed email in Module 2, or any of the survey tools in Module 3. The takeaway is that all functionality should be tested in all browsers for compatibility and working access. Directions should also be strengthened for the use of these tools, for example, if an account is needed to be created, and if there is something specific the student needs to do to make sure the tool is accessible to them.
Question: Was the overall site easy to use and navigate? All respondents felt the site was easy to navigate.
Question: How long did it take you to finish the instruction (minus completing the activities)? Instruction took between 5 and 30 minutes and was not module specific.
Question: The Instruction on the website was effective in helping me complete activities.
All agreed instruction was effective in helping to complete activities.
Question: It was easy to learn with this website. All checked TRUE that it was easy to learn from the website.
Question: I feel like each module lined up with the instructional goals. All checked TRUE that each module successfully lined up with the instructional goals.
Question: I like the way the information on this website was arranged. All checked TRUE I like the way the information on this website was arranged.
Question: Specify which activity or activities did you completed.
The respondents chose different activities. One reported the Appropriate Work Attire did not work.
Question: Is there anything on the website that does not make sense? If so what?
One respondent was unable to upload their work as instructed in the module site. All others reported positively that the website made sense. One did not answer.
Question: How would you improve this website's usability?
Some respondents did not understand the word usability and this was clear because they responded that all students who need to improve interpersonal soft skills should use the website. This is not what we were asking about with the use of term usability. Two respondents were unsure and replied yes - giving no detail how they would change usability and one did not respond. One reported on usability thoughtfully by responding, “For some of the activities there were too many tabs to open and go back and forth to.” It would have strengthened our feedback to know specifically which module this defect was discovered in.
Question: What changes would you make to the curriculum?
When asked how they would change the curriculum we received the following suggestions: “For the pink hair dress code one, don’t use that picture use something with out of order clothes.” Another responded “I like the curriculum but I liked module 3 best because it had a lot of very good information and resources adding, “Maybe just add more content.” One student responded N/A, suggesting they didn’t think the word ‘curriculum’ applied to them. This may mean we should have asked the question using more colloquial language.
Question: What specifically do you like about this website?
In response to specifically what you like about this website we received the following responses: “i like how you can complete all of these activities to help students know how to be professional in a work force”; “It asked questions and made me think.”; “the videos”; ”Every thing”; ”I like module 3, I found the videos really interesting”; “I enjoyed looking at the content” and ”It was easy to follow and understand.”
Question: What specifically did you not like about this website? In response to What specifically did you not like about this website we received the following input: there wasn’t anything i disliked about the website, How does of the links didn’t work, so i could not move forward, I didn’t like that some modules had more content than others, it felt unbalanced. I didn’t like that some modules had more content than others, it felt unbalanced. That I couldn’t use the notes website thing, Nothing and one N/A.
Question: What is at least one fact or idea that you learned from this website about interpersonal skills and making a good impression in the business world?
In response to What is at least one fact or idea that you learned from this website about interpersonal skills and making a good impression in the business world? We received the following: appearance is an important factor when going into the workforce. it can give people perspective on how you act and your personality, Dress appropriately and use manners, and a good attitude., email etiquette, To be nice,That being able to read people in the business world is key to being successful and professional, That you should have a good work attire or people will think that you won’t be able to perform your job very well, That you should have a good work attire or people will think that you won’t be able to perform your job very well. Not to use exclamation marks in a professional email
Question: Overall, I enjoyed this website. All agreed Overall, I enjoyed this website.
From a collaboration standpoint, working as a team to develop the site, we struggled with a site template and consistency in our modules, which some end users noted. The lesson learned here is to develop a site template which details how each module will be constructed very specifically, including the tools from weebly, like the survey, that will be utilized in building the module and then stick with it, even if there are multiple content authors. As a team we could have been more consistent in the way we developed the site across all three modules. Another aspect of this assignment that it was difficult to navigate, as a team, exactly what multimedia artifacts needed to be created and how they should be designed overall in the context of the site. Each module author developed their own content and delivery, but taken as an overall site, it is clear that we, as module authors, had individual visions, and it is not clear if overall the vision matched what we delivered. It is also important to note that the original blueprint of the instruction was a more brick and mortar approach to instruction, and this plan had to be modified for delivery in a website, multimedia platform. It may have made more sense to have had one team member serve as the site developer. In the current limited summer time constraint of this course, that was not a viable option.
In conclusion, our site received positive suggestions and good feedback that we can use to make it even better. It was a good experience to approach the development of a site from the ADDIE instructional design model and to work together to incorporate into our design useful content to improve the interpersonal skills of high school seniors.