Leading Online Professional Development for Instructional Technology Coaches with Effective Design Elements

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This article presents design elements of online professional development for instructional technology coaches. A single case study of 30 coaches who participated in a PD program shows which design elements were beneficial, challenging, and the development of TPACK by participants.

This manuscript presents an interesting topic and the focus on online PD is timely due to the recent need after the COVID pandemic. The authors explain well different design elements that are important for the design of efficient PD. I would recommend revising several parts of the manuscript to better connect the introduction with the methodology and discussions as follows:

-The abstract could include the number of participants in the PD program

-The introduction includes interesting information however the presentation of the content is sometimes repetitive. For example, the design elements (content focus, active learning, coherence...) have been repeated several types. I would suggest summarizing information and also presenting studies concerning participants' perspectives on the different design elements.

-Some paragraphs include a few sentences and many paragraphs should be reformulated or merged.

-The research questions are not presented before the methodology. The logical structure is to derive some RQs based on the theoretical background.

-Multiple types of data were collected but it is not very clear how this data was analyzed. How many people were involved in data analysis? How did researchers analyzed the interviews. There should be more information about the data analysis.

-In the results sections, a graph could show how the responses from Likert scale items were developed after each training session. In addition, the main topics regarding the qualitative data analysis could be highlighted. One option could be to present a table with the key topics identified from the analysis and the corresponding evidence (e.g., interviews etc). Sample qualitative data need also to be presented.

-More information concerning TPACK and PD could be presented in the introduction so that it aligns with the current study. How TPACK was operationalized in the study is not clear.

-Further work needs to be done in the discussion with relevant literature.  The limitations and strengths of the study need to be discussed. 

Shallow comments:

-It is not clear what is CDC.

-Does veteran mean experts in this context?

Overall, the manuscript describes important information about design elements of online PD and an interesting case study. However, it requires improvement. The presentation of relevant studies about PD and participants' perceptions, TPACK studies and the concrete comments explained above could help to revise the manuscript.

 

Comments on the Quality of English Language

I do not have comments on the quality of English but on the paragraph structure because many of them are very short. The manuscript would also benefit from English language proof reading

Author Response

Thank you for your suggestions. We heavily edited the manuscript. The large revisions are highlighted in yellow on the manuscript to allow for easy identification of where we edited to address the reviewers suggestions. Hopefully we met all of your revision suggestions and the manuscript is easier to understand as a reader.

Attached is a Word document that details how we addressed your suggestions.

Thank you again for your thorough review!

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review your article. After a thorough review of your article, “Leading Online Professional Development for Instructional Coaches with Effective Design Elements,” it is evident that you have invested considerable time, effort, and expertise in its development. I offer the following feedback to help improve the manuscript:

 

Abstract

-       Some phases are written as capitalized in one part of the abstract and then written as lower case in other parts of the abstract:

o   Instructional Technology Coaches

o   Pathways

-       Data is plural so need to make the following phrase plural:

o   Line 10: “Qualitative data was collected…” change to “Qualitative data were collected…”

-       Clarify how Adult Learning Theory (mentioned in the keywords) informed the study.

Introduction

-       Line 32-33 – The following sentence seems out of place: “Technology leaders within the school building also needed to find an online solution to provide collaboration and support digital learning for both students and educators.” The topic of the article is about online professional development for technology coaches, so it seems out of place to discuss online solutions for students.

-       Line 58 – “Online professional development design elements have just begun to be researched due to the pandemic.” Readers could disagree with this statement. Online formats for professional development were accessible prior to the pandemic, so it might be good to include some examples, such as MOOCs; however, the pandemic was a catalyst for increasing accessibility to professional development through online formats.

-       Line 61 – “By addressing this gap, designers will be able to design and conduct successful online professional developments.” It seems there is an assumption that if professional development is designed using a particular set of design elements, then that will turn into a successful professional development offering. Designing professional learning is an important part as is “conducting” or facilitating online professional development. Is there a source that can support these claims?

-       The introduction will need revision to help improve the argument for conducting the study and recognizing that online professional development is not new.

 

Literature Review

-       Add an introduction to the literature review and help the reader understand how the literature reviewed informed the study. Consider how you might organize the literature and articulate the aim of the literature section and help define key terms used.

Section 2.1

-       How does professional development in PK-12 schools support the study? It seems this study is about the professional development for the technology coaches, so why is there discussion about professional development in PK-12?  Are these technology coaches also teachers?

-       The section (Lines 68-83) need to be organized into coherent paragraphs. Review each paragraph to make sure each one has a topic sentence/main idea that is identifiable, followed by supportive key points and details with literature sources, and a concluding sentence. Provide more depth around the ideas presented and define the terms used (e.g., five-level evaluation framework, design and structure of PD, major goals, design elements…).

-       Improve clarity through active voice - Line 71: “This led to a shift towards creating a conceptual framework for effective PD.” What was “this” and who created the framework?

Section 2.2

-       This section discusses in-person professional development design elements – clarify why this is included in the review? Are you suggesting elements of design for in-person also apply to online professional development? Include critiques about in-person professional development and literature that shows there is limited impact.

Section 2.3

This section discusses blended professional development design elements. Additional citations are needed to identify the sources for the claims. Examples of some of the statements that need citations:

o   Key design principles focus on participant engagement and active learning.

o   Participants are encouraged to make decisions on their learning path, with a variety of online resources catering to different difficulty levels and learning styles.

o   A supportive accountability structure is created by allowing participants to provide feedback and interact with the same small group throughout the PD.

o   Relationships formed within these groups are strengthen during in-person sessions

Section 2.4

This seems to be the most relevant section for the literature review. I’m not convinced the other sections are needed. I think it’s important to define PD and then discuss design elements as informed by the literature. Review this section carefully as I see similar issues with paragraph coherence, missing citations for claims, and use of passive voice. It is also important to include critiques for online PD in this section.

 

Add information about how the paper’s argument is built on an appropriate base of theory. A section could be added to discuss the Adult Learning Theory and TPACK framework used for the study. There is limited information about this in the methodology section.

 

Methodology

-       Discuss case study methodology and include citations

-       Clarify how information was gathered from the coaches to determine the “coaches represented a diverse range of characteristics, including varying levels of TPACK.” How was TPACK knowledge evaluated?

-       Add more detail about the Pathways Program – how many synchronous sessions? Topics/themes included in the sessions? Who reviewed/evaluated  assignments? Who was involved in the collaborative design? How was TPACK acquisition evaluated?

-       Add more detail about when the interviews and surveys took place (did these occur during the Pathways Program?) and how did these help answer research questions? It sounds like post-session surveys helped inform subsequent sessions. What part (design, organization, content…)?

-       More detail is needed about the methods used and the analysis for each method. Consider adding sample questions used in the instruments or providing in the appendix (interview questions before the program, interview questions during the programs, post-interview questions, survey questions, rubrics, classwork artifacts); add information about the likert scale options. How many interviews were conducted? Include amounts for all the data sources.

-       What is the feedback? Was this another instrument or does this refer to specific survey questions? Please clarify.

-       Describe process for analysis and include citations

 

It is difficult to determine if the results are presented clearly and analyzed appropriately as the foundational parts of the article need considerable revision and more detail. It seems that participants were provided with feedback forms and asked what elements were most effective. It is unclear how participants’ perceptions correlate to “key design elements in effective only professional development.” Collaboration is mentioned as a key design element. Who was collaborating and how during the Pathways Program? What were the participants sharing with peers and during breakout sessions? How is this connected to definitions of collaboration?

 

It is difficult to see how the data supports the results. Here’s one example:

“Three participants highlighted the importance of coaching and expert support. Aria, for instance, appreciated the detailed explanation of the Teams folder, which she had not used expensively before.” I’m not sure how this is coaching. It sounds like information was provided to the participant about functionality. Add more detail about the coaching that occurred.

 

Consider how the data could be integrated and provided more cohesively for each of the questions. It may also be helpful to present some of the data visually in tables (e.g., how the cohort showed an increase in the evaluated areas of TPACK).

 

Also, review citation/reference format for consistency throughout the article.

 

I hope this detailed feedback will help strengthen the article.

Comments on the Quality of English Language

The above feedback also provides suggestions for improving the quality of English language in the paper related to coherence of paragraphs, use of active voice, adding detail to clarify information presented.

Author Response

Thank you for your suggestions. We heavily edited the manuscript. The large revisions are highlighted in yellow on the manuscript to allow for easy identification of where we edited to address the reviewers suggestions. Hopefully we met all of your revision suggestions and the manuscript is easier to understand as a reader.

Attached is a Word document that details how we addressed your suggestions.

Thank you again for your thorough review!

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for implementing modifications in the manuscript. The paper has been improved based on the previous version. I understand the contributions of the study, however, there are also limitations such as the low number of participants, the single case study. The results are also mainly descriptive that is why the key points need to be summarized based on the research questions. I recommend to further revise the paper based on the following points:

-It would be better to nclude the RQs in a later stage (after the literature review), as they should be based on the previous research studies.

-The title of Table 1 could be revised. 

-The titles of Table 2-3 look similar. They can be revised so that it is clear the difference in their content. Moreover, information in Table 2-3 are repeated.

-Table 7 needs to be included in the results section or include the information as a text in the discussion. 

-Limitations need to be further explained. For example, the single-case study design and the number of participants have certain limitations.  

-In some sections, the information needs to be further summarized.  

Author Response

• We had other suggestions that we needed to put the RQ's earlier in the paper and include it in the introduction.
• Titles to Tables 1,2,3 were revised
• We included more text in the discussion regarding Table 7.
• We added additional information about the limitations.

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review your article again. I can see that considerable revisions have been made.

The literature review section could use some further edits. Some of the sections (in-person and blended PD) are not relevant to the study and could be removed.  Any elements that are relevant for online PD could be included in that section of the review. 

The assumptions in the TPACK section are a bit confusing. It seems that Technology Coaches are assumed to have a high level of TK because of their role. It is unclear why the Technology Coaches "should be able to demonstrate significant growth" within the TPACK framework. There seems to be an assumption that they have low levels is PK and CK and would then be able to demonstrate growth. However, these are teachers so wouldn't they also have expertise in PK and CK so then growth would not be as significant?

Review the text to make sure terms are consistent throughout. For example, "instructional technology leaders" - are these the same as coaches?

Who are the leaders? Are these leaders of the PD? Same as researchers?

What does it mean that the participants for this study were contracted? 

The additional sections in the methodology and research context and appendix provide sufficient detail.

Table 1 - What does "completed All Data" mean? List the methods.

The findings sections need some organization. Perhaps some subheadings could help to organize the results. It does not seem there was much revision done in this section. The tables are helpful additions.

Comments on the Quality of English Language

The paper will need review and copyediting. 

Author Response

• We added clarifying statements in the LR section to help with the explanation of why the in-person and blended design elements sections impact the information provided in the online design elements.
• We provided clarification in the TPACK sections.
• Provided explanation that instructional technology leaders are different from instructional technology coaches.
• Provided explanation for contracted employees.
• Additional information provided in Table 1 for "All Data".
• The findings section added some edits.