A Methodological Framework Based on a Quantitative Assessment of New Technologies to Boost the Interoperability of Railways Services

Question

1. What is the aim of the study?

2. What methodology was used for assessing the acceptability of new IT solutions in public transport, and what are its key features?

3. What are Operational KPIs?

4. What is the satisfaction index of travelers belonging to profile vector 'r' with the new IT service 'j' offered by TSP 'k'?

Accepted Answer

The aim of the study is to present a methodological framework to assess, in a quantitative way, how innovative technologies can respond to the needs of tourists and Transport Service Providers (TSP) involved in the digital ecosystem for door-to-door travel in Europe, making railways and public transport more attractive. The study assesses the needs of travelers belonging to different socio-demographic profiles and TSPs based on rail transport in a methodological way, addressing societal trends such as a reduction in Greenhouse Gas (GHG) emissions and road congestion. The concept of the attractiveness of rail and public transport depends on complex psychological factors from a scientific and technical perspective. The assessment methodology consolidates the concept of 'user profile' and the ability of the system to respond to needs and expectations, including sociodemographic-related aspects such as aging, reduced mobility, impairments, and other specific conditions. The effectiveness of IT solutions, adaptability of data to satisfying the requirements of both travelers and operators, equity of ITs deployed in society, and potential acceptance by the market and railway operators are considered in the assessment methodology introduced in this work. The methodology belongs to the framework developed in the first phase of the H2020 Shift2Rail IP4MaaS project, which sets six demo sites where this methodology will be applied in the second phase.

Accepted Answer

The methodology used for assessing the acceptability of new IT solutions in public transport combines several decision-making methodologies, including the Analytic Hierarchy Process (AHP), SWOT analysis, and risk management plan. It also incorporates qualitative research approaches, such as case studies and interviews, to gather insights from experts and real users. The methodology focuses on evaluating operational Key Performance Indicators (KPIs) and User Satisfaction Index (USI) to measure the benefits and satisfaction levels of users. It aims to provide a comprehensive and structured decision-making approach that reflects real-world conditions and ensures equal and fair opportunities for all users to access and use new technologies in public transport.

Accepted Answer

Operational KPIs are quantitative and objective indicators that measure the benefits of new IT services for specific TSPs and traveler profiles. They are automatically collected through the application of new technology in the field. Each KPI is measured in a specific unit and converted into a dimensionless KPI, with a value between 0 and 1, by dividing it by the maximum value measured for this KPI among all TSPs. A higher dimensionless value indicates better performance. KPIs are defined based on the technological capabilities of TSPs for integrating new IT services, and they are used to evaluate the effectiveness of different travel solutions and means of transport.

Accepted Answer

The satisfaction index of travelers belonging to profile vector 'r' with the new IT service 'j' offered by TSP 'k' is calculated as: USI Traveller rjk = m rjk w=1 n 1jk +n 2jk r v=1 Score question wv EQUATION where Score question wv is the score given to the question 'v' by the respondent 'w'; n 1jk is the number of questions applicable to all the profiles measuring satisfaction with the new IT service 'j' offered by the TSP 'k'; n 2jk r is the number of questions applicable only to profile 'r', measuring satisfaction with the IT service 'j' offered by the TSP 'k'; m rjk is the number of respondents to the USI questionnaire belonging to the profile 'r', measuring the satisfaction with the IT service 'j' offered by the TSP 'k'. The satisfaction index of a TSP 'k' regarding an IT service 'j' is calculated as: EQUATION where 'Score question' is the score given to the question number 'v' and 'n' is the number of questions in the USI questionnaire belonging to a specific IT service 'j' offered by the TSP 'k'; m rjk is the number of respondents to the USI questionnaire measuring the satisfaction with the IT service 'j' offered to the TSP 'k'. In both equations, a 5 is used to normalize and obtain a value between 0.2 and 1, because the answer to each question has a value between 1 (representing the minimum satisfaction) and 5 (representing the maximum satisfaction) [11] . Regarding the satisfaction level found in USI surveys, one of the options is N/A (not applicable).If the traveler or TSP has not tested a specific IT service, the 'Score question wv ' value from 'v = 1' to 'n 1jk + n 2jk r ' for the traveller or the 'Score question v ' value from 'v = 1' to 'n j ' for the TSP will be set as zero..

Accepted Answer

IT service effectiveness is calculated using the equation EQUATION, where d Traveller and d TSP are determined based on USI Traveler and USI TSP values. The effectiveness is calculated by summing the values of operational KPIs, USI Travelers, and USI TSPs with equal weight. The result is a dimensionless value between 0.2 and 1, where a higher score indicates better effectiveness. Different demonstration scenarios can be compared to analyze how the needs of travelers in different locations or demo sites are matched by the same innovative technology offered by different TSPs. The linear distribution of weights among KPIs and USIs in the effectiveness formula could be a limitation, and refinements may be made in the future. Effectiveness is used as a quantitative metric to assess the benefits of new technologies for users, tourists, and TSPs, and low values indicate areas for improvement to meet user needs and expectations. It also helps in boosting the interoperability of railway services by creating more equitable IT services.

Accepted Answer

Regression analysis can identify pairwise correlations between variables, allowing for fixed connections in Bayesian network analysis. This leads to more accurate results by determining correlations among variables in a network based on USI questionnaires and operational KPIs. By using regression analysis, researchers can enhance their methodology by incorporating additional data analysis tools to gain deeper insights into the relationships between variables and their impact on the research outcomes.

Accepted Answer

AS-IS scenarios describe current travel experiences in a demo site, while TO-BE scenarios show how technologies in the IP4 ecosystem could overcome current limitations. They are defined through a user journey template, highlighting flow, steps, actions, personal preferences, and decisions users make during travel. These scenarios help identify existing problems, areas of improvement, and potential use-cases for IP4MaaS demonstrations. In the Athens case study, AS-IS scenarios were used to understand current travel experiences, and TO-BE scenarios were used to envision improved travel experiences with IP4MaaS technologies. The scenarios were collected from 17 travelers willing to test IT innovations for tourists in Athens.

Accepted Answer

In the USI Travellers' Survey, profiles were categorized using a coding method. The profiles were divided into five groups based on specific characteristics. Group 1 (r = 1) included all profiles. Group 2 (r = 2) consisted of unemployed people, low-income individuals, retired individuals, and students. Group 3 (r = 3) comprised disabled or impaired individuals, people with physical or mental illnesses, individuals in wheelchairs, people with reduced mobility, and those with visual and hearing impairments. Group 4 (r = 4) represented the elderly. Lastly, Group 5 (r = 5) included women. These profiles were chosen for sentiment analysis due to their heightened sensitivity to satisfaction regarding the survey's functionalities, as indicated in the study [43].

Accepted Answer

The User Satisfaction Index (USI) values for all traveler profiles are similar to specific profiles such as low-income people, disabled people, the elderly, and women. However, slight differences in satisfaction are observed when tourists belong to these specific profiles. The USI survey for travelers used in this case study can be consulted in Appendix A. In total, the number of responding USI travelers was seventeen. Table 2 addresses the USI for travelers, along with codes and their meanings and values. Despite the methodology's ability to consider different specific profiles, no data regarding low-income (r = 2) and disabled people (r = 3) were collected through the surveys, resulting in no values assigned for these two profiles in Table 2.

Accepted Answer

In Table 4, the most effective profiles are 'general' profiles, with Location-Based Experience provided by TSPs 'OASA' and 'MIRAKLIO' having values of 0.92 and 0.82, respectively, out of 1. These values indicate the highest effectiveness among the profiles and TSPs considered in the study. The table illustrates the effectiveness values derived for the profile vector, functionality, and TSP, providing insights into the performance of different profiles and TSPs in terms of effectiveness. The effectiveness values are calculated using the effectiveness formula, which takes into account various factors such as USI TSPs, USI Travelers, and KPIs value. The results presented in Table 4 help researchers and stakeholders understand the impact of different profiles and TSPs on the overall effectiveness of the system or service being studied.

Accepted Answer

High correlations exist between tourist satisfaction and journey planning variables. Specifically, general satisfaction from journey planning (J1K1q1) is strongly correlated with cost-saving through journey planning (J1K1q3) and Location-Based Experience (J8K1q3). Additionally, general satisfaction related to Location-Based Experience (J8K1q1) is correlated with journey planning satisfaction (J1K1q1) and time-saving and cost-saving associated with Location-Based Experience (J8K1q2 and J8K1q3). These correlations highlight the importance of costs and commercial offers in influencing tourist satisfaction and decision-making during journey planning.

Accepted Answer

The ANOVA test results in Table 8 show significant differences in satisfaction related to specific functionalities of TSP OASA based on sociodemographic variables. For example, the Location-Based Experience functionality shows significant differences between urban and non-urban areas, while the journey planning functionality shows significant differences between people with and without disabilities. These differences are attributed to the performance and aim of the respective functionalities, such as presenting commercial offers or entertainment services, and providing accessible routes for disabled individuals. The significance of these differences is determined by the p-value, with values less than or equal to 0.05 indicating significance.

Accepted Answer

Insufficient sample size can negatively impact data analysis accuracy by leading to decreased precision and reliability. When the response sample size is not sufficient, it may not adequately represent different sociodemographic profiles, resulting in biased or skewed results. This limitation can affect the validity of the study's findings and limit the generalizability of the results. To ensure accurate and precise data analysis, it is crucial to achieve a sufficient sample size for both general and specific profiles. This allows for a more comprehensive understanding of the population being studied and enhances the credibility of the research findings. Therefore, researchers should carefully consider the required sample size for each module of the regression analysis, Bayesian network analysis, and ANOVA test to ensure the reliability and validity of their data analysis.

Accepted Answer

The Effectiveness index is a dimensionless index that ranges between 0.2 and 1, where a higher score indicates better performance. It measures how well an innovative technology offered by a TSP matches the needs and expectations of its users, travelers, and TSPs. The index is calculated using two quantitative types of data, Operational KPIs and USIs, which allow for the assessment of the Effectiveness of a specific innovative technology. In the research, the top three values of Effectiveness were related to the Location-Based Experience functionality provided by OASA and MIRAKLIO, and the journey planning functionality provided by OASA. This methodology enables further data analyses, such as regression analysis, Bayesian networks, and ANOVA, to gain insights into the impact of innovative IT functionalities on the number of tourists and travelers using railways in a post-COVID-19 environment.

A Methodological Framework Based on a Quantitative Assessment of New Technologies to Boost the Interoperability of Railways Services

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What is the aim of the study?

2. What methodology was used for assessing the acceptability of new IT solutions in public transport, and what are its key features?

3. What are Operational KPIs?

4. What is the satisfaction index of travelers belonging to profile vector 'r' with the new IT service 'j' offered by TSP 'k'?

5. How is IT service effectiveness calculated?

6. How can regression analysis extend methodology?

7. What are AS-IS and TO-BE scenarios?

8. How were profiles categorized in the USI Travellers' Survey?

9. What is the User Satisfaction Index (USI) for different traveler profiles?

10. What are the most effective profiles and TSPs in Table 4?

11. What correlations exist between tourist satisfaction and journey planning variables?

12. What is the significance of ANOVA test results in Table 8?

13. How does insufficient sample size affect data analysis accuracy?

14. What is the Effectiveness index and how does it measure the needs and expectations of users, travelers, and TSPs?

Citations

An Assessment Methodology about the Effectiveness of Mobility IT Solutions: Application to Six Demo Sites

Boosting Public Transport Through Innovative IT Solutions that Match the Needs and Expectations of All Stakeholders

References

The focused interview

A Practical Guide to Focus-Group Research

Developing Indicators for Comprehensive and Sustainable Transport Planning

Behaviour Theory and Soft Transport Policy Measures

Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood

Related Papers (5)

Validation of timing properties for interoperability in distributed real-time applications.

Intelligent algorithms and standards for interoperability in Internet of Things

Synchronizing Data through Update Queries in Interoperable E-Health and Technology Enhanced Learning Data Sharing Systems.

Citizens, seamlessness, and care - inter-relationships and inter-operability.

Performance Evaluation of Interoperable Micro-clouds