From: Daniel Jackson Date: Thu Jun 06, 2002 11:28:11 AM US/Eastern To: Heinz Erzberger , Russ Paielli , Michelle Eshow Cc: tsafe@geyer.lcs.mit.edu Subject: TSAFE questions dear heinz, russ and michelle, many thanks for spending time with me last week discussing TSAFE. it was very helpful. this note is an attempt to write down some of the things you told me (or rather that i heard!) so that if i've got confused you can put me right. there are also some questions that occurred to me on my flight back. 1. critical maneuver detection. the main topic we discussed was a new idea (described in the paper recently written by heinz and russ, 'concept for next generation air-traffic control') that addresses in particular a class of errors that have been prominent in russ's analysis of operational errors, in which a pilot fails to make a critical maneuver, bringing the aircraft into conflict with another. the idea is to extrapolate the expected trajectory past the point at which it is expected to turn, and compute conflicts for the extrapolated segment. if such a conflict is found, a 'critical maneuver warning' is generated telling the pilot that there will be a conflict unless the critical maneuver is taken. if the pilot fails to make the maneuver, and passes the point at which the expected trajectory diverges from the extrapolated segment, the conflict becomes a standard 'predicted' conflict (although perhaps it should be flagged as more serious since the pilot has blundered?). we discussed for a long time what it meant to extrapolate. heinz described the extrapolation as the 'steady state' path that the aircraft would follow if no changes were made. the notion of 'no changes' is not a physical one, but depends on the extent of automation and the pilot's perception of what constitutes intervention. i asked in particular a question that i think michael had raised earlier about whether the steady state of an aircraft banking (say in a holding pattern) includes the banking itself (so that the extrapolation would be a circle), or not (in which case the extrapolation would be a tangent). heinz told me that banking is considered an intervention, and the period in which the plane is turning is considered to be a single transition, even if under the control of a flight management system. so banking is not a steady state. we then turned to altitude changes. heinz's initial explanation of critical maneuvers suggested that the extrapolation of a descending plane should assume a fixed descent rate (and not a fixed altitude), so that the act of descending is not an intervention, and the descent is steady state. we got in a big confusion about this because controllers and pilots view altitude changes as transitions, but we came to the conclusion (i think) that this was just an unfortunate clash of terminology, and we would have to consider levelling off after descent to to be a transition, and the descent itself to be steady state. also, heinz said that we shouldn't worry about turns in holding patterns -- that they're complicated and can be ignored for now. we should view an expected trajectory -- indeed any trajectory -- as a sequence of points, each adjacent pair connected by a straight line segment, smoothed into a curve at the join point (in order to analyze precisely when a critical maneuver should occur). 2. conlfict alert. we talked a bit about conflict detection in the host computer. russ's operational error scenarios show that this facility is sometimes effective at averting conflicts, although TSAFE would produce a warning longer in advance. the conflict detection algorithm (NAS-MD-321, 14.0 Conflict Alert) is quite complicated and ad hoc, but seems to work well given its limitations. there is complicated notion of 'turn intent' used to dead reckon accurately from a turning plane in the absence of information about bank angle. 3. no transgression zone. heinz explained that another form of analysis that is needed in TSAFE is to warn pilots when an unexpected maneuver would cause a conflict in the very short term. for example, when two aircraft fly in parallel with a small vertical separation, a descent by the upper one would violate the 'no transgression zone' of the lower one. TSAFE should therefore produce a warning akin to the critical maneuver warning saying 'don't turn!' or 'don't descend!'. the two warnings are duals; critical maneuver warnings are about errors of omission (what happens if you fail to do something active); no transgression warnings are about errors of commission (what happens if you do something you weren't supposed to do). 4. ETMS. we talked about the suitability of ETMS as a data source for our TSAFE prototype. heinz said he thought it would be usable, although because the data rate is so much lower than the host computer feed (1 minute rather than 12 seconds), it might not be possible to do any reasonable altitude analysis, which would likely cause most operational errors to be missed. he also warned us that ETMS's altitude data was for a long time based on assigned and not actual altitude (from Mode-C transponder), but that this was supposed to have been fixed in most centers. we may need to pick a center that does not have this problem. cleveland would be a more interesting center than boston because it has lots of cross traffic, en route traffic and congestion. since heinz and russ will be using cleveland data for their TSAFE prototype, it might also give us a basis of comparison. detecting actual altitude change rates may be hard from ETMS data, but assigned altitudes are provided as explicit flight plan amendment messages. 5. off-route detection. for the purpose of our prototype, heinz suggested that for off-route detection it would suffice to determine whether an aircraft had strayed beyond 4 nmi either side of the expected trajectory, and whether its heading was within some bounds. if an aircraft is off-route, we should simply compute a dead reckoning trajectory, and not do more complicated things (such as attempting to predict whether it will rejoin the flight plan trajectory). 6. monitoring clearance and state. my discussion with russ, looking over his operational error scenarios, and reading the paper, made me wonder that we should perhaps not think of TSAFE as checking the state of the world (aircraft states with respect to each other and to clearances), but rather as checking aircraft state and clearances as two rather different things. perhaps we should view the issuing of a clearance as an event to be checked immediately, but view aircraft state as something that gets checked periodically, at a time determined by TSAFE and not the presence of radar input. 7. a few comments and questions on the paper: - Figure 1. we've discussed this one before, but it's still not resolved in my mind. what happens at the point at which AACS, TSAFE and the controller interface are all shown as mutually connected? this is potentially a big problem, compromising the independence of TSAFE. i'm very uncomfortable with the idea (that you mention later) of having TSAFE use trajectories from AACS, since this would likely cause AACS failures to infect TSAFE also. - you describe the inherent limitation of CTAS vis a vis safety in terms of the finite set of solutions that can be built into software. what you don't mention, which is perhaps more important, is that whatever function CTAS is supposed to perform, it is unlikely to perform without error in practice. it's just not possible to verify a million lines of code. - what is the time horizon of TSAFE? in one place you say "about 3 minutes", another "1-2 minutes". - Figure 2 (TSAFE Architecture) shows the key functions (conflict detection, critical maneuver detection, etc), but the text describes them as "modules". this is premature; it's unlikely that a good software design would be organized into modules along functional lines. - the discussion of critical maneuver warnings suggests that a message be sent a few minutes before the planned maneuver to the pilot. in a computerized design, it should probably be sent repeatedly, starting a few minutes in advance, to guard against message loss. - you say that TCAS detects collisions rather than conflicts. is this really a fundamental difference beyond time scale? i've marked some typos up in my copy and will send them to you if you're interested. 8. combination conditions. how do you regard combinations of conditions? for example, is the intersection of two trajectory extrapolations cause for generating a critical maneuver warning? this warning would mean that if _both_ aircraft failed to execute the maneuver there would be a problem. 9. off-route analysis. what is the status of the expected (flight plan) trajectory when an aircraft is off route? assuming we dead reckon as you suggested, should we just ignore the expected trajectory? 10. no transgression maneuvers. how do we generate candidate maneuvers for the no-transgression zone analysis? and how exactly is the no-transgression zone defined? /daniel ======================================================================== From: Heinz Erzberger Date: Thu Jun 06, 2002 01:12:24 PM US/Eastern To: Daniel Jackson Cc: Rpaielli@mail.arc.nasa.gov, Meshow@mail.arc.nasa.gov Subject: Re: TSAFE questions Daniel, Thank you for your extremely insightful comments and questions. Russ and I will respond in detail after we have fully digested your memo. In general, your comments indicate an accurate and comprehensive understanding of our current thinking and of our discussion. Give us a few days to reply more fully. I would be grateful if you sent me the typos you found in the paper. Thanks, Heinz ======================================================================== From: Russ Paielli Date: Mon Jun 10, 2002 01:11:18 PM US/Eastern To: Daniel Jackson Cc: Heinz Erzberger , Michelle Eshow , tsafe@geyer.lcs.mit.edu Subject: Re: TSAFE questions Daniel Jackson wrote: > dear heinz, russ and michelle, > > many thanks for spending time with me last week discussing TSAFE. it was > very helpful. this note is an attempt to write down some of the things > you told me (or rather that i heard!) so that if i've got confused you > can put me right. there are also some questions that occurred to me on > my flight back. > Thanks for stopping by. I'll try to address some of your concerns below. > > 1. critical maneuver detection. the main topic we discussed was a new > idea (described in the paper recently written by heinz and russ, > 'concept for next generation air-traffic control') that addresses in > particular a class of errors that have been prominent in russ's analysis > of operational errors, in which a pilot fails to make a critical > maneuver, bringing the aircraft into conflict with another. the idea is > to extrapolate the expected trajectory past the point at which it is > expected to turn, and compute conflicts for the extrapolated segment. if > such a conflict is found, a 'critical maneuver warning' is generated > telling the pilot that there will be a conflict unless the critical > maneuver is taken. if the pilot fails to make the maneuver, and passes > the point at which the expected trajectory diverges from the > extrapolated segment, the conflict becomes a standard 'predicted' > conflict (although perhaps it should be flagged as more serious since > the pilot has blundered?). Once the critical maneuver is missed, a conflict should be imminent. For example, if a descending aircraft is supposed to level off 1000 feet above a cruising one, the conflict will begin as soon as the leveling maneuver is missed. A couple of our operational error cases were like that. In other cases, however, the conflict might not occur quite that fast but it will still be within a couple of minutes. > > we discussed for a long time what it meant to extrapolate. heinz > described the extrapolation as the 'steady state' path that the aircraft > would follow if no changes were made. the notion of 'no changes' is not > a physical one, but depends on the extent of automation and the pilot's > perception of what constitutes intervention. i asked in particular a > question that i think michael had raised earlier about whether the > steady state of an aircraft banking (say in a holding pattern) includes > the banking itself (so that the extrapolation would be a circle), or not > (in which case the extrapolation would be a tangent). heinz told me that > banking is considered an intervention, and the period in which the plane > is turning is considered to be a single transition, even if under the > control of a flight management system. so banking is not a steady state. > we then turned to altitude changes. heinz's initial explanation of > critical maneuvers suggested that the extrapolation of a descending > plane should assume a fixed descent rate (and not a fixed altitude), so > that the act of descending is not an intervention, and the descent is > steady state. we got in a big confusion about this because controllers > and pilots view altitude changes as transitions, but we came to the > conclusion (i think) that this was just an unfortunate clash of > terminology, and we would have to consider levelling off after descent > to to be a transition, and the descent itself to be steady state. I think of "steady state" as constant velocity. That includes cruising at constant speed and heading, of course. It also includes climbing or descending at a constant velocity. Actually, airplanes tend to climb and descend at a constant Mach or CAS (calibrated airspeed), and both true airspeed and groundspeed vary (even with no winds). However, that is close enough to constant velocity for our purposes here. Yes, we should be careful not get confused by mixing terminology from different disciplines. Aerodynamicists probably don't consider a steady climb or descent as "steady-state" flight, but for our purposes I think it is. Also, a controller probably thinks of an altitude change as a single "maneuver" because it involves only a single command or clearance. For our purposes, however, a change from flying level at one altitude to flying level at another altitude involves two maneuvers: (1) the beginning of descent (or climb) and (2) the leveling off. > > also, heinz said that we shouldn't worry about turns in holding > patterns > > -- that they're complicated and can be ignored for now. we > > should view an expected trajectory -- indeed any trajectory -- as a > > sequence of points, each adjacent pair connected by a straight line > > segment, smoothed into a curve at the join point (in order to analyze > > precisely when a critical maneuver should occur). > > I think holding patterns should probably be considered a special class. They are "steady state" even though it is not constant velocity. We may need to give this more thought later. > > 2. conlfict alert. we talked a bit about conflict detection in the host > computer. russ's operational error scenarios show that this facility is > sometimes effective at averting conflicts, although TSAFE would produce > a warning longer in advance. the conflict detection algorithm > (NAS-MD-321, 14.0 Conflict Alert) is quite complicated and ad hoc, but > seems to work well given its limitations. there is complicated notion of > 'turn intent' used to dead reckon accurately from a turning plane in the > absence of information about bank angle. It remains to be seen how much earlier TSAFE can detect conflicts compared to CA. But don't forget that in the Automated or Advanced Airspace concept of the future, intent will be known much more accurately and tracking will be much more accurately, with a higher bandwidth. That should allow TSAFE to perform much better. I don't think CA is designed to make use of detailed intent information to the extent that TSAFE will ultimately do. Also, CA does not provide resolution advisories as TSAFE will do. > > 3. no transgression zone. heinz explained that another form of analysis > that is needed in TSAFE is to warn pilots when an unexpected maneuver > would cause a conflict in the very short term. for example, when two > aircraft fly in parallel with a small vertical separation, a descent by > the upper one would violate the 'no transgression zone' of the lower > one. TSAFE should therefore produce a warning akin to the critical > maneuver warning saying 'don't turn!' or 'don't descend!'. the two > warnings are duals; critical maneuver warnings are about errors of > omission (what happens if you fail to do something active); no > transgression warnings are about errors of commission (what happens if > you do something you weren't supposed to do). That's right. Actually, I think the term "no-transgression zone" can be applied to either case, but that's just a matter of semantics that needs to be worked out. In one case, you want to make sure the aircraft maneuvers as expected; in the other case you want to make sure the aircraft does NOT maneuver unexpectedly. > > 4. ETMS. we talked about the suitability of ETMS as a data source for > our TSAFE prototype. heinz said he thought it would be usable, although > because the data rate is so much lower than the host computer feed (1 > minute rather than 12 seconds), it might not be possible to do any > reasonable altitude analysis, which would likely cause most operational > errors to be missed. he also warned us that ETMS's altitude data was for > a long time based on assigned and not actual altitude (from Mode-C > transponder), but that this was supposed to have been fixed in most > centers. we may need to pick a center that does not have this problem. > cleveland would be a more interesting center than boston because it has > lots of cross traffic, en route traffic and congestion. since heinz and > russ will be using cleveland data for their TSAFE prototype, it might > also give us a basis of comparison. detecting actual altitude change > rates may be hard from ETMS data, but assigned altitudes are provided as > explicit flight plan amendment messages. I would be a bit concerned about the slow update rate of ETMS. Yes, you can do a lot with it, but you might get a bit frustrated down the road when you want to do more. But then I don't know what your alternative is. I'll defer to Heinz on this one. > > 5. off-route detection. for the purpose of our prototype, heinz > suggested that for off-route detection it would suffice to determine > whether an aircraft had strayed beyond 4 nmi either side of the expected > trajectory, and whether its heading was within some bounds. if an > aircraft is off-route, we should simply compute a dead reckoning > trajectory, and not do more complicated things (such as attempting to > predict whether it will rejoin the flight plan trajectory). I would consider the dead-reckoning projection as a first cut. When the aircraft is off track and its intent is unknown, you are really playing a guessing game. What you need to do eventually, I think, is to try to cover all reasonable potential trajectories. The dead-reckoning (constant velocity) path is just one path. Another possibility is that the aircraft will try to get back on track. Then there's everything in between. If the heading is way off, say 45 degrees for example, then you need to consider all the possible ways the aircraft might try to get back on track. I don't know if there is an elegant way to do that. I think we may want to "fan" the heading in increments of, say, 10 or 15 degrees, or perhaps a region of airspace should just be "blocked out." This area needs more thought. > > 6. monitoring clearance and state. my discussion with russ, looking over > his operational error scenarios, and reading the paper, made me wonder > that we should perhaps not think of TSAFE as checking the state of the > world (aircraft states with respect to each other and to clearances), > but rather as checking aircraft state and clearances as two rather > different things. perhaps we should view the issuing of a clearance as > an event to be checked immediately, but view aircraft state as something > that gets checked periodically, at a time determined by TSAFE and not > the presence of radar input. I'm not sure exactly what you are getting at here. I agree, however, that altitude clearances should be checked as soon as possible after they are entered by the controller. As I explained to you last week, four of our nine operational error cases were caused by faulty altitude clearances. That is, the controller actually CAUSED the conflict by telling an airplane to climb or descend at a bad time. TSAFE should have been able to detect those cases and stop the command from being issued. That check should be done as quickly as possible. > > 7. a few comments and questions on the paper: > > - Figure 1. we've discussed this one before, but it's still not resolved > in my mind. what happens at the point at which AACS, TSAFE and the > controller interface are all shown as mutually connected? this is > potentially a big problem, compromising the independence of TSAFE. i'm > very uncomfortable with the idea (that you mention later) of having > TSAFE use trajectories from AACS, since this would likely cause AACS > failures to infect TSAFE also. > The AACS generates the trajectories, and TSAFE then verifies that the trajectories are free of conflicts, and it monitors for conformance to the trajectories. TSAFE will detect AACS errors, but it cannot function independently of AACS, at least not if I understand it correctly. TSAFE will depend on the nominal operation of AACS. That could be a problem, and perhaps we need to give more thought to what TSAFE should do in the case of a major failure of AACS. Is that what you are getting at? > > - you describe the inherent limitation of CTAS vis a vis safety in terms > of the finite set of solutions that can be built into software. what you > don't mention, which is perhaps more important, is that whatever > function CTAS is supposed to perform, it is unlikely to perform without > error in practice. it's just not possible to verify a million lines of > code. Agreed. > > - what is the time horizon of TSAFE? in one place you say "about 3 > minutes", another "1-2 minutes". At this point, I think it's a parameter that could be anywhere from 2 to 4 minutes. > > - Figure 2 (TSAFE Architecture) shows the key functions (conflict > detection, critical maneuver detection, etc), but the text describes > them as "modules". this is premature; it's unlikely that a good software > design would be organized into modules along functional lines. You know more about that than we do. It depends, of course, on your definition of "module." I read somewhere that a software module is basically a source file (and its corresponding object file), but I don't know if that definition is widely accepted. It could also mean an executable process, I suppose. What is your definition of a module? > > - the discussion of critical maneuver warnings suggests that a message > be sent a few minutes before the planned maneuver to the pilot. in a > computerized design, it should probably be sent repeatedly, starting a > few minutes in advance, to guard against message loss. > > - you say that TCAS detects collisions rather than conflicts. is this > really a fundamental difference beyond time scale? > Yes it is. TCAS doesn't care about loss of 5 mile separation. We have a nice FAA pamphlet on TCAS version 7 that you might be interested in. > > i've marked some typos up in my copy and will send them to you if you're > interested. Thanks. > > 8. combination conditions. how do you regard combinations of conditions? > for example, is the intersection of two trajectory extrapolations cause > for generating a critical maneuver warning? this warning would mean that > if _both_ aircraft failed to execute the maneuver there would be a > problem. > > 9. off-route analysis. what is the status of the expected (flight plan) > trajectory when an aircraft is off route? assuming we dead reckon as you > suggested, should we just ignore the expected trajectory? > No. See above. > > 10. no transgression maneuvers. how do we generate candidate maneuvers > for the no-transgression zone analysis? and how exactly is the > no-transgression zone defined? That is still being worked out. > /daniel Russ