A MEASURE OF HUMAN-INTEGRATED SYSTEM PERFORMANCE UNDER TIME-VARYING CIRCUMSTANCES by Nguyen-Vang-Phuc Nguyen A Dissertation Submitted to the Faculty of Purdue University In Partial Fulfillment of the Requirements for the degree of Doctor of Philosophy School of Industrial Engineering West Lafayette, Indiana May 2018 ProQuest Number: 10808798 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. ProQuest 10808798 Published by ProQuest LLC (2018 ).
Copyright of the Dissertation is held by the Author. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code Microform Edition © ProQuest LLC. 789 East Eisenhower Parkway P.
Box 1346 Ann Arbor, MI 48106 - 1346 ii THE PURDUE UNIVERSITY GRADUATE SCHOOL STATEMENT OF COMMITTEE APPROVAL Dr. Landry, Chair Department of Industrial Engineering Dr. Sara McComb Department of Industrial Engineering Dr. Karen Marais School of Aeronautics and Astronautics Dr.
Denny Yu Department of Industrial Engineering Approved by: Dr. Abhijit Deshmukh Head of the Departmental Graduate Program ii Dedicated to the memory of my parents, And in honor of my aunt and my sister, All for their love and support ii ACKNOWLEDGMENTS I could not have completed this work without the support of many. First and foremost, I would like to express my special thanks to my advisor, Dr. Steve Landry, for his guidance, advice and support throughout my PhD life.
Landry inspired me to explore a new approach in the field of human factors, and I owe a debt of gratitude to him for his time and careful guidance. I also would like to thank my committee members, Drs. Sara McComb, Karen Marais and Denny Yu for their tremendous help. I would like to express my thanks to Dr.
McComb who always encourages me and supports me. Additionally, I would like to thank Dr. Marais and Dr. Yu for appreciating my research and patiently encouraging me to improve my weak aspects.
I owe a deep debt of appreciation to VEF (Vietnam Education Foundation) for giving me a fellowship opportunity to study at one of the nation's leading research institutions. To take this opportunity, I’d like to extend warm thanks to my VEF friends who have shown interest and given encouragement. I am deeply grateful to my peers at Purdue University have provided wonderful support and special thanks go to Yul Kwon, Julian Archer and Harsh Aggarwal. They checked on me, empathized with me, and offered help.
I am also thankful to my friend, Loc Nguyen, for his unconditional friendship and for supporting me when I need it. None of this would have been possible without the love and support of my family. My parents, especially my mom, instilled in me the significance of education at a very young age and was always giving me the freedom to make my life choices. I am particularly grateful for my aunt and my sister, who are supportive in every way and have taken care of my family when I have been absent from home.
Lastly, I would like to show my appreciation to master Minh-Niem who taught me the simple but insightful lessons of self-balancing and living in true happiness. His lessons have changed my thought patterns and my views on many subjects of life. iii TABLE OF CONTENTS LIST OF TABLES. viii LIST OF FIGURES.
x LIST OF ABBREVIATIONS .1 Overview of learning curves .1 Learning curve history.2 Concept of learning curves .3 Concept of learning .4 Log-linear models.2 Estimating the learning rate .3 Learning improvement, learning rate and progress ratio.2 Levels of learning rates .4 Individual and organizational learnings .5 Graphical presentations of human performance with learning curves .6 Family of learning curves models .7 Applications of Learning curves .8 Learning curves with time-varying circumstances. THEORETICAL WORK: THE ADAPTABILITY MODEL .4 Measures of adaptability .5 Learning index of human operators tested in a system .6 Adaptability index of human operators tested in a system design .7 Effectiveness of a system design (EDu/PS) .8 Usability/preference subject score (SS) .5 System design evaluation regarding adaptability parameters .6 Some cases of adaptability parameters. EXPERIMENTAL WORK: DEMONSTRATION STUDY. EXPERIMENTAL WORK: EMPIRICAL EXPERIMENT .3 Purposes and research questions .3 Experimental design table.
RESULTS AND DISCUSSION .1 Part 1: Visual examination of data .1 Learning curves under time varying circumstances .2 Calculations on the adaptability parameters .4 Differences on learning rates .2 Part 2: Statistical examination of data .3 Part 3: Calculations on Performance score and usability/preference score.1 Performance score/ Effectiveness .2 Usability/preference subject score. CONCLUSIONS AND RECOMMENDATION FOR FUTURE WORK .2 Limitations and Recommendation for Future Work. LITERATURE REVIEW: FAMILY OF LEARNING CURVE MODELS. LITERATURE REVIEW: TYPICAL LEARNING CURVE APPLICATIONS.
CLASSIFICATION OF ADAPTABILITY COEFFICICENT. SOME PATTERNS OF ADAPTABILITY PARAMETERS. VALIDITY PROCEDURE FOR THE SETTINGS OF DEMONSTRATION EXPERIMENT. LITERATURE REVIEW: VALIDITIES IN RESEARCH PROCESS.
DATA ANALYSIS OF DEMONSTRATION STUDY. RESEARCH PARTICIPANT CONSENT FORM. INSPECTION WEB-FORM. STANDARD COLOR CHART.
RAW DATA COLLECTION. CALCULATIONS OF LEARNING CURVES. CALCULATIONS ON ADAPTABILITY PARAMETERS. CALCULATIONS OF PS.
CALCULATIONS OF SS. FITTING LEARNING SLOPES. GENERAL LINEAR MODEL OF GAMMA-BAR AND BETA-BAR VERSUS SYSTEM, ORDER, DELAY. 261 viii LIST OF TABLES Table 2.1 The meaning of y and x are different under two concepts (Crawford, 1944; Liao, 1988) .2 Cumulative average vs.
cumulative average time (Brookfield, 2005) .3 The relationship between the machine- labor and learning rate .4 Levels of typical learning rates .1 System designs with adaptability parameters .2 Some cases of the adaptability parameters in time varying circumstances .1 Setting values of experiment conditions .2 Settings of the demonstration study .3 Learning slopes and learning indexes .4 Adaptability coefficients and adaptability indexes .5 Design D1: Performance score (PS), b and .2 Orders of Circumstances .4 Comparison of different metrics for two systems .1 Formulas used for calculating adaptability parameters .3 Converted learning rate .4 Differences |ri-rj| where i: row, j: column .5 The differences between learning rates regarding the ratio of human/machine in task design .6 Fitting gamma-bar in distributions .7 Fitting Beta-bar in distributions .8 Hypotheses on the effects of order and delay .9 Result of GLM on Gamma-bar versus System, Order, Delay .10 Result of GLM on Beta-bar versus System, Order, Delay .11 Test for Equal Variances of Gamma-bar .12 Test for Equal Variances of Beta-bar .13 Kruskal-Wallis Test of Gamma-bar .14 Kruskal-Wallis Test of Beta-bar .15 Performance score/ Effectiveness of System I-iPhone .17 Performance score/ Effectiveness of System II-Tablet.19 Post-Study System Usability Questionnaire (PSSUQ) Scores .20 Scores of two systems .1 Fitted learning slopes for iPhone .2 Fitted learning slopes for Tablet. 235 x LIST OF FIGURES Figure 1.1 Contexts of circumstances. (a) The context of discrete circumstances, (b) The context of continuous series of time-varying circumstances. The point down arrow () indicates a direction that a system design can be tested under a circumstance or a series of circumstances.
The right point arrow () stands for the continuation of circumstantial occurrences.2 An illustration of time-varying circumstances. Within a circumstance, the conditions are unchanging or consistent within the boundary of this circumstance. Ri stands for the ith repetition in a circumstance. Ci stands for the condition in a circumstance.
The condition Ci only changes in a new circumstance.3 Slopes of a human-performance curve .4 Human performance in individual circumstances (a), and in time-varying circumstances (b). The curves in figure a) describe human performance in discrete circumstances. There are no relations or connections among these curves; The curves in Figure b) describe human performances in continuous series of time-varying circumstances .1 Learning curve with learning period and time .2 A basic learning curve .3 An illustration of different types of the learning curve y = Tx(b) from difficult tasks to easy tasks (left-right and top-down) at Y-axes scale from 0 to 12 .4 Log-linear learning curve .5 Log-log learning curve .6 A presentation of cumulative average and unit curves on the log-log coordinate .7 Unit data plotted for setting concrete floor planks (Thomas et al.8 Cumulative average plot for setting concrete floor planks (Thomas et al.10 The uniform rates (90%, 80%, 70%) of learning curves (Yelle, 1979).11 Interpreting the learning curves: short-term losses vs. long-term gains .12 A conventional log-linear learning curve .13 Bottlenecks affect the learning curve .15 The learning curve of the worker A .16 The learning curve of the worker B .17 Labor time per unit vs.
unit number represents the cumulative average performance curve.18 Output per time period vs. time represents the industry learning curve .19 Learning curves of novices and skills operators.20 The concept of human performance in time-varying circumstances.1 The concept of human performance in time-varying circumstances.2 An illustration of the cumulative number of task-units completed in c=4 time-varying circumstances .3 The lower- and upper-bound ranges in four time-varying circumstances .4 Adaptability coefficients of an operator from CIRi to CIRi+1 .1 The search-and-isolate task .2 Four time-varying circumstances .3 Design 1 – Twin Arm type (Tamiya 3ch Radio Control Robot Construction Set) .4 An experiment setting for the demonstration study .5 Searching area for the demonstration study .6 Learning curve of human-integrated robot system (Subject 1-4) in four time-varying circumstances. The gloves are RUCPAC Professional Tech brand with weather resistant and touchscreen compatible features.3 Cases of colored balls .4 Subjects perform the inspection task in circumstance OG (outside with gloves) with iPhone (left) and tablet (right).5 The subject performs the inspection task in circumstance IN (inside and no-gloves) 71 Figure 6.1 Learning curves of subjects using System I – iPhone .2 Learning curves of subjects using System II - Tablet .3 Learning curves of System I vs. System II on the same scale of y-axis from 0 to1000 seconds .4 Interval plot and boxplot of vs.5 Interval plot and boxplot of vs.6 Interval plot and boxplot of b vs order .7 Interval plot and boxplot of b vs delay .8 Probability plots of fitted distributions for gamma-bar (a-d) .9 Probability plots of fitted distributions for beta-bar (a, b, c) .10 Residual plots for gamma-bar .11 Residual plots for beta-bar.
100 xiii LIST OF ABBREVIATIONS Ai(x) Cumulative average performance time of each of the x cumulative task-units completed in time-varying circumstances CIR Circumstance Ci Condition i {C1=Inside} Condition inside {C2=Outside}, Condition outside {C3=Gloves}, Condition with gloves {C4=No-gloves} Condition without gloves or no-gloves {IN}, {OG}, {ON} Individual circumstances xiv ABSTRACT Author: Nguyen-Vang-Phuc, Nguyen. Institution: Purdue University Degree Received: May 2018 Title: A Measure of Human-Integrated System Performance under Time-Varying Circumstances Major Professor: Steven Landry There are many methods to evaluate a system from given options in discrete or fixed situations (‘circumstance’). However, most systems are operated under time-varying circumstances and it’s not known how to evaluate the best system design when the operator in that system moves between time varying circumstances. In this dissertation, an adaptability model has been formalized using symbolic notion, which is based on learning curve theory and the adaptability measures are proposed.
In the first study (‘the demonstration study’), the measures proved that they could be calculated and the learning curves could be plotted in continuous varying-circumstances. In the second study (‘the empirical study’), we tested two systems under three varying-circumstances. The primary purpose of this experiment was to study whether the order and delay of changing circumstances affect the adaptability measures, in which influential circumstances are randomly arranged. The statistical tests showed that order and delay do not have effects on adaptability measures.
However, the results from the graphical analysis provide useful information to adjust the setting of circumstances regarding the levels of order and delay factors in practice.