31 Object persistence and databases inrepiciomeas reating and manipulating an number of objects.What happens to these objects when the current execution terminates? Transient objects will disappear with the current session;but many applications also need persistent objects,which will stay around from session to session.Persistent objects may need to be shared by several applications,raising the need for databases. In this overview of persistence issues and solutions we will examine the three approaches that O-O developers have at their disposal for manipulating persistent objects. They can rely on persistence mechanisms from the programming language and development environment to get object structures to and from permanent storage.They can combine object technology with databases of the most commonly available kind(not O-O):relational databases.Or they can use one of the newer object-oriented database systems,which undertake to transpose to databases the basic ideas of object technology. This chapter describes these techniques in turn,providing an overview of the technology of O-O databases with emphasis on two of the best-known products.It ends with a more futuristic discussion of the fate of database ideas in an O-O context. 31.1 PERSISTENCE FROM THE LANGUAGE For many persistence needs it suffices to have,associated with the development environment,a set of mechanisms for storing objects in files and retrieving them from files.For simple objects such as integers and characters,we can use input-output facilities similar to those of traditional programming. Storing and retrieving object structures See“Deep storage: As soon as composite objects enter the picture,it is not sufficient to store and retrieve a first view ofpersis- individual objects since they may contain references to other objects,and an object tence",page 250. deprived of its dependents would be inconsistent.This observation led us in an earlier chapter to the Persistence Closure principle,stating that any storage and retrieval mechanism must handle,together with an object,all its direct and indirect dependents.The following figure served to illustrate the issue:
31 Object persistence and databases Executing an object-oriented application means creating and manipulating a certain number of objects. What happens to these objects when the current execution terminates? Transient objects will disappear with the current session; but many applications also need persistent objects, which will stay around from session to session. Persistent objects may need to be shared by several applications, raising the need for databases. In this overview of persistence issues and solutions we will examine the three approaches that O-O developers have at their disposal for manipulating persistent objects. They can rely on persistence mechanisms from the programming language and development environment to get object structures to and from permanent storage. They can combine object technology with databases of the most commonly available kind (not O-O): relational databases. Or they can use one of the newer object-oriented database systems, which undertake to transpose to databases the basic ideas of object technology. This chapter describes these techniques in turn, providing an overview of the technology of O-O databases with emphasis on two of the best-known products. It ends with a more futuristic discussion of the fate of database ideas in an O-O context. 31.1 PERSISTENCE FROM THE LANGUAGE For many persistence needs it suffices to have, associated with the development environment, a set of mechanisms for storing objects in files and retrieving them from files. For simple objects such as integers and characters, we can use input-output facilities similar to those of traditional programming. Storing and retrieving object structures As soon as composite objects enter the picture, it is not sufficient to store and retrieve individual objects since they may contain references to other objects, and an object deprived of its dependents would be inconsistent. This observation led us in an earlier chapter to the Persistence Closure principle, stating that any storage and retrieval mechanism must handle, together with an object, all its direct and indirect dependents. The following figure served to illustrate the issue: See “Deep storage: a first view of persistence”, page 250
1038 OBJECT PERSISTENCE AND DATABASES $31.1 The need for persistence name "Almaviva" closure landlord loved_one ▲(PERSONI) 02 037 name "Figaro" "Susanna" name landlord landlord loved_one loved_one (PERSONI) (PERSONI) The Persistence Closure principle stated that any mechanism that stores Ol must also store all the objects to which it refers,directly or indirectly;otherwise when you retrieve the structure you would get a meaningless value ("dangling reference")in the loved one field for O1. We saw the mechanisms of class STORABLE which provide the corresponding facilities:store to store an object structure and retrieved to access it back.This is a precious mechanism,whose presence in an O-O environment is by itself a major advantage over traditional environments.The earlier discussion gave a typical example of use:implementing the SAVE facility of an editor.Here is another,from ISE's own practice.Our compiler performs several passes on representations of the software text. The first pass creates an internal representation,known as an Abstract Syntax Tree(AST). Roughly speaking,the task of the subsequent passes is to add more and more semantic information to the AST (to "decorate the tree")until there is enough to generate the compiler's target code.Each pass finishes by a store;the next pass starts by retrieving the AST through retrieved. The STORABLE mechanism works not only on files but also on network connections such as sockets;it indeed lies at the basis of the Net client-server library. Storable format variants Procedure store has several variants.One,basic store,stores objects to be retrieved by the same system running on the same machine architecture,as part of the same execution or ofa later one.These assumptions make it possible to use the most compact format possible for representing objects. Another variant,independent store,removes all these assumptions;the object representation is platform-independent and system-independent.It consequently takes a little more space,since it must use a portable data representation for floating-point and other numerical values,and must include some elementary information about the classes of the system.But it is precious for client-server systems,which must exchange
1038 OBJECT PERSISTENCE AND DATABASES §31.1 The Persistence Closure principle stated that any mechanism that stores O1 must also store all the objects to which it refers, directly or indirectly; otherwise when you retrieve the structure you would get a meaningless value (“dangling reference”) in the loved_one field for O1. We saw the mechanisms of class STORABLE which provide the corresponding facilities: store to store an object structure and retrieved to access it back. This is a precious mechanism, whose presence in an O-O environment is by itself a major advantage over traditional environments. The earlier discussion gave a typical example of use: implementing the SAVE facility of an editor. Here is another, from ISE’s own practice. Our compiler performs several passes on representations of the software text. The first pass creates an internal representation, known as an Abstract Syntax Tree (AST). Roughly speaking, the task of the subsequent passes is to add more and more semantic information to the AST (to “decorate the tree”) until there is enough to generate the compiler’s target code. Each pass finishes by a store; the next pass starts by retrieving the AST through retrieved. The STORABLE mechanism works not only on files but also on network connections such as sockets; it indeed lies at the basis of the Net client-server library. Storable format variants Procedure store has several variants. One, basic_store, stores objects to be retrieved by the same system running on the same machine architecture, as part of the same execution or of a later one. These assumptions make it possible to use the most compact format possible for representing objects. Another variant, independent_store, removes all these assumptions; the object representation is platform-independent and system-independent. It consequently takes a little more space, since it must use a portable data representation for floating-point and other numerical values, and must include some elementary information about the classes of the system. But it is precious for client-server systems, which must exchange (PERSON1) name "Almaviva" landlord loved_one (PERSON1) name "Figaro" landlord loved_one (PERSON1) "Susanna" name landlord loved_one O1 O2 O3 The need for persistence closure
$31.2 BEYOND PERSISTENCE CLOSURE 1039 potentially large and complex collections of objects among machines of widely different architectures,running entirely different systems.For example a workstation server and a PC client can run two different applications and communicate through the Net library, with the server application performing the fundamental computations and the client application taking care of the user interface thanks to a graphical library such as Vision. Note that the storing part is the only one to require several procedures-basic store, independent store.Even though the implementation of retrieval is different for each format,you will always use a single feature retrieved,whose implementation will detect the format actually used by the file or network data being retrieved,and will automatically apply the appropriate retrieval algorithm. 31.2 BEYOND PERSISTENCE CLOSURE The Persistence Closure principle is,in theory,applicable to all forms of persistence.It makes it possible,as we saw,to preserve the consistency of objects stored and retrieved. In some practical cases,however,you may need to adapt the data structure before letting it be applied by mechanisms such as STORABLE or the O-0 database tools reviewed later in this chapter.Otherwise you may end up storing more than you want. The problem arises in particular because of shared structures,as in this setup: Small structure CITY structure with reference to big shared address structure FAMILY structure A relatively small data structure needs to be archived.Because it contains one or more references to a large shared structure,the Persistence Closure principle requires archiving that structure too.In some cases you may not want this.For example,as illustrated by the figure,you could be doing some genealogical research,or other processing on objects representing persons;a person object might,through an address field,reference a much bigger set of objects representing geographical information.A similar situation occurs in ISE's ArchiText product,which enables users to manipulate structured documents,such as programs or specifications.Each document,like the FAMILY structure in the figure, contains a reference to a structure representing the underlying grammar,playing the role ofthe C/TY structure;we may want to store a document but not the grammar,which already exists elsewhere and may be shared by many documents
§31.2 BEYOND PERSISTENCE CLOSURE 1039 potentially large and complex collections of objects among machines of widely different architectures, running entirely different systems. For example a workstation server and a PC client can run two different applications and communicate through the Net library, with the server application performing the fundamental computations and the client application taking care of the user interface thanks to a graphical library such as Vision. Note that the storing part is the only one to require several procedures — basic_store, independent_store. Even though the implementation of retrieval is different for each format, you will always use a single feature retrieved, whose implementation will detect the format actually used by the file or network data being retrieved, and will automatically apply the appropriate retrieval algorithm. 31.2 BEYOND PERSISTENCE CLOSURE The Persistence Closure principle is, in theory, applicable to all forms of persistence. It makes it possible, as we saw, to preserve the consistency of objects stored and retrieved. In some practical cases, however, you may need to adapt the data structure before letting it be applied by mechanisms such as STORABLE or the O-O database tools reviewed later in this chapter. Otherwise you may end up storing more than you want. The problem arises in particular because of shared structures, as in this setup: A relatively small data structure needs to be archived. Because it contains one or more references to a large shared structure, the Persistence Closure principle requires archiving that structure too. In some cases you may not want this. For example, as illustrated by the figure, you could be doing some genealogical research, or other processing on objects representing persons; a person object might, through an address field, reference a much bigger set of objects representing geographical information. A similar situation occurs in ISE’s ArchiText product, which enables users to manipulate structured documents, such as programs or specifications. Each document, like the FAMILY structure in the figure, contains a reference to a structure representing the underlying grammar, playing the role of the CITY structure; we may want to store a document but not the grammar, which already exists elsewhere and may be shared by many documents. address FAMILY structure CITY structure Small structure with reference to big shared structure
1040 OBJECT PERSISTENCE AND DATABASES $31.2 In such cases you may want to"cut out"the references to the shared structure before storing the referring structure.This is,however,a delicate process.First,you must as always make sure that at retrieval time the objects will still be consistent-satisfy their invariants. But there is also a practical problem:to avoid complication and errors,you do not really want to modify the original structure;only in the stored version should references be cut out. Once again the techniques of object-oriented software construction provide an See“Deferred elegant solution,based on the ideas of behavior class reviewed in the discussion of classes as partial inheritance.One of the versions of the storing procedure,custom independent store,has implementations: the notion of behav- the same effect as independent store by default,but also lets any descendant of a library iorc☑ass”,page504. class ACTIONABLE redefine a number of procedures which do nothing by default,such as pre store which will be executed just before an object is stored and post store which will be executed after.So you can for example have pre store perform preserve;address :Void where preserve,also a feature of ACT/ONABLE,copies the object safely somewhere. Then post action will perform a call to restore which restores the object from the preserved copy. For this common case it is in fact possible to obtain the same effect through a call of the form store ignore ("address") where ignore takes a field name as argument.Since the implementation of store ignore may simply skip the field,avoiding the two-way copy of preserve and restore,it will be more efficient in this case,but the pre store-post store mechanism is more general, allowing any actions before and after storage.Again,you must make sure that these actions will not adversely affect the objects. You may in fact use a similar mechanism to remove an inconsistency problem arising at retrieval time;it suffices to redefine the procedure post retrieve which will be executed just before the retrieved object rejoins the community of approved objects.For example an application might redefine post retrieve,in the appropriate class inheriting from ACT/ONABLE,to execute something like address:=my_city_structure.address_value (... hence making the object presentable again before it has had the opportunity to violate its class invariant or any informal consistency constraint. There are clearly some rules associated with the ACT/ONABLE mechanism;in particular,pre store must not perform any change of the data structure unless post store corrects it immediately thereafter.You must also make sure that post retrieve will perform the necessary actions (often the same as those of post store)to correct any inconsistency introduced into the stored structure by pre store.Used under these rules,the mechanism lets you remain faithful to the spirit of the Persistent Closure principle while making its application more flexible
1040 OBJECT PERSISTENCE AND DATABASES §31.2 In such cases you may want to “cut out” the references to the shared structure before storing the referring structure. This is, however, a delicate process. First, you must as always make sure that at retrieval time the objects will still be consistent — satisfy their invariants. But there is also a practical problem: to avoid complication and errors, you do not really want to modify the original structure; only in the stored version should references be cut out. Once again the techniques of object-oriented software construction provide an elegant solution, based on the ideas of behavior class reviewed in the discussion of inheritance. One of the versions of the storing procedure, custom_independent_store, has the same effect as independent_store by default, but also lets any descendant of a library class ACTIONABLE redefine a number of procedures which do nothing by default, such as pre_store which will be executed just before an object is stored and post_store which will be executed after. So you can for example have pre_store perform preserve; address := Void where preserve, also a feature of ACTIONABLE, copies the object safely somewhere. Then post_action will perform a call to restore which restores the object from the preserved copy. For this common case it is in fact possible to obtain the same effect through a call of the form store_ignore ("address") where ignore takes a field name as argument. Since the implementation of store_ignore may simply skip the field, avoiding the two-way copy of preserve and restore, it will be more efficient in this case, but the pre_store-post_store mechanism is more general, allowing any actions before and after storage. Again, you must make sure that these actions will not adversely affect the objects. You may in fact use a similar mechanism to remove an inconsistency problem arising at retrieval time; it suffices to redefine the procedure post_retrieve which will be executed just before the retrieved object rejoins the community of approved objects. For example an application might redefine post_retrieve, in the appropriate class inheriting from ACTIONABLE, to execute something like address := my_city_structure ● address_value (…) hence making the object presentable again before it has had the opportunity to violate its class invariant or any informal consistency constraint. There are clearly some rules associated with the ACTIONABLE mechanism; in particular, pre_store must not perform any change of the data structure unless post_store corrects it immediately thereafter. You must also make sure that post_retrieve will perform the necessary actions (often the same as those of post_store) to correct any inconsistency introduced into the stored structure by pre_store. Used under these rules, the mechanism lets you remain faithful to the spirit of the Persistent Closure principle while making its application more flexible. See “Deferred classes as partial implementations: the notion of behavior class”, page 504
$31.3 SCHEMA EVOLUTION 1041 31.3 SCHEMA EVOLUTION A general issue arises in all approaches to O-O persistence.Classes can change.What if you change a class of which instances exist somewhere in a persistent store?This is known as the schema evolution problem. The word schema comes from the relational database world,where it describes the architecture of a database:its set of relations (as defined in the next section)with,for every relation,what we would call its type-number of fields and type of each.In an O-O context the schema will also be the set of types,given here by the classes. Although some development environments and database systems have provided interesting tools for O-O schema evolution,none has yet provided a fully satisfactory solution.Let us define the components of a comprehensive approach. An object's generat- Some precise terminology will be useful.Schema evolution occurs if at least one ing class(or genera-class used by a system that attempts to retrieve some objects (the retrieving system) tor)is the class of which it is a direct differs from its counterpart in the system that stored these objects(the storing system). instance.See"Basic Object retrieval mismatch,or just object mismatch for short,occurs when the retrieving form",page 219. system actually retrieves a particular object whose own generating class was different in the storing system.Object mismatch is an individual consequence,for one particular object,of the general phenomenon of schema evolution for one or more classes. Remember that in spite of the terms“storing system”and“retrieving system”this whole discussion is applicable not only to storage and retrieval using files or databases,but also to object transmission over a network,as with the Net library.In such a case the more accurate terms would be“sending system”and“receiving system”. Exercise E31.1.page To keep the discussion simple,we will make the usual assumption that a software 1062,asks you to system does not change while it is being executed.This means in particular that all the study the conse- instances of a class stored by a particular system execution refer to the same version of the quences of removing this assumption. class;so at retrieval time either all of them will produce an object mismatch,or none of them will.This assumption is not too restrictive;note in particular that it does not rule out the case of a database that contains instances ofmany different versions of the same class, produced by different system executions. Naive approaches We can rule out two extreme approaches to schema evolution: You might be tempted to forsake previously stored objects (schema revolution!). The developers of the new application will like the idea,which makes their life so much easier.But the users of the application will not be amused You may offer a migration path from old format to new,requiring a one-time,en masse conversion of old objects.Although this solution may be acceptable in some cases,it will not do for a large persistent store or one that must be available continuously. What we really need is a way to convert old objects on the fly as they are retrieved or updated.This is the most general solution,and the only one considered in the rest of this discussion
§31.3 SCHEMA EVOLUTION 1041 31.3 SCHEMA EVOLUTION A general issue arises in all approaches to O-O persistence. Classes can change. What if you change a class of which instances exist somewhere in a persistent store? This is known as the schema evolution problem. The word schema comes from the relational database world, where it describes the architecture of a database: its set of relations (as defined in the next section) with, for every relation, what we would call its type — number of fields and type of each. In an O-O context the schema will also be the set of types, given here by the classes. Although some development environments and database systems have provided interesting tools for O-O schema evolution, none has yet provided a fully satisfactory solution. Let us define the components of a comprehensive approach. Some precise terminology will be useful. Schema evolution occurs if at least one class used by a system that attempts to retrieve some objects (the retrieving system) differs from its counterpart in the system that stored these objects (the storing system). Object retrieval mismatch, or just object mismatch for short, occurs when the retrieving system actually retrieves a particular object whose own generating class was different in the storing system. Object mismatch is an individual consequence, for one particular object, of the general phenomenon of schema evolution for one or more classes. Remember that in spite of the terms “storing system” and “retrieving system” this whole discussion is applicable not only to storage and retrieval using files or databases, but also to object transmission over a network, as with the Net library. In such a case the more accurate terms would be “sending system” and “receiving system”. To keep the discussion simple, we will make the usual assumption that a software system does not change while it is being executed. This means in particular that all the instances of a class stored by a particular system execution refer to the same version of the class; so at retrieval time either all of them will produce an object mismatch, or none of them will. This assumption is not too restrictive; note in particular that it does not rule out the case of a database that contains instances of many different versions of the same class, produced by different system executions. Naïve approaches We can rule out two extreme approaches to schema evolution: • You might be tempted to forsake previously stored objects (schema revolution!). The developers of the new application will like the idea, which makes their life so much easier. But the users of the application will not be amused. • You may offer a migration path from old format to new, requiring a one-time, en masse conversion of old objects. Although this solution may be acceptable in some cases, it will not do for a large persistent store or one that must be available continuously. What we really need is a way to convert old objects on the fly as they are retrieved or updated. This is the most general solution, and the only one considered in the rest of this discussion. An object’s generating class (or generator) is the class of which it is a direct instance. See “Basic form”, page 219. Exercise E31.1, page 1062, asks you to study the consequences of removing this assumption
1042 OBJECT PERSISTENCE AND DATABASES $31.3 Ifyou happen to need en-masse conversion,an on-the-fly mechanism will trivially let you do it:simply write a small system that retrieves all the existing objects using the new classes,applying on-the-fly conversion as needed,and stores everything. On-the-fly object conversion The mechanics of on-the-fly conversion can be tricky;we must be particularly careful to get the details right,lest we end up with corrupted objects and corrupted databases. First,an application that retrieves an object and has a different version of its generating class may not have the rights to update the stored objects,which may be just as well since other applications may still use the old version.This is not,however,a new problem.What counts is that the objects manipulated by the application be consistent with their own class descriptions;an on-the-fly conversion mechanism will ensure this property.Whether to write back the converted object to the database is a separate question -a classical question of access privilege,which arises as soon as several applications,or even several sessions of the same application,can access the same persistent data. Database systems,object-oriented or not,have proposed various solutions Regardless of write-back aspects,the newer and perhaps more challenging problem is how each application will deal with an obsolete object.Schema evolution involves three separate issues-detection,notification and correction: Detection is the task of catching object mismatches (cases in which a retrieved object is obsolete)at retrieval time. Notification is the task of making the retrieving system aware of the object mismatch,so that it will be able to react appropriately,rather than continuing with an inconsistent object(a likely cause of major trouble ahead!) Correction is the task,for the retrieving system,of bringing the mismatched object to a consistent state that will make it a correct instance of the new version of its class -a citizen,or at least a permanent resident,of its system of adoption. All three problems are delicate.Fortunately,it is possible to address them separately. Detection We can define two general categories of detection policy:nominal and structural. In both cases the problem is to detect a mismatch between two versions of an object's generating class:the version used by the system that stored the object,and the version used by the system which retrieves it. In the nominal approach,each class version is identified by a version name.This assumes some kind of registration mechanism,which may have two variants: If you are using a configuration management system,you can register each new version of the class and get a version name in return (or specify the version name yourself)
1042 OBJECT PERSISTENCE AND DATABASES §31.3 If you happen to need en-masse conversion, an on-the-fly mechanism will trivially let you do it: simply write a small system that retrieves all the existing objects using the new classes, applying on-the-fly conversion as needed, and stores everything. On-the-fly object conversion The mechanics of on-the-fly conversion can be tricky; we must be particularly careful to get the details right, lest we end up with corrupted objects and corrupted databases. First, an application that retrieves an object and has a different version of its generating class may not have the rights to update the stored objects, which may be just as well since other applications may still use the old version. This is not, however, a new problem. What counts is that the objects manipulated by the application be consistent with their own class descriptions; an on-the-fly conversion mechanism will ensure this property. Whether to write back the converted object to the database is a separate question — a classical question of access privilege, which arises as soon as several applications, or even several sessions of the same application, can access the same persistent data. Database systems, object-oriented or not, have proposed various solutions Regardless of write-back aspects, the newer and perhaps more challenging problem is how each application will deal with an obsolete object. Schema evolution involves three separate issues — detection, notification and correction: • Detection is the task of catching object mismatches (cases in which a retrieved object is obsolete) at retrieval time. • Notification is the task of making the retrieving system aware of the object mismatch, so that it will be able to react appropriately, rather than continuing with an inconsistent object (a likely cause of major trouble ahead!). • Correction is the task, for the retrieving system, of bringing the mismatched object to a consistent state that will make it a correct instance of the new version of its class — a citizen, or at least a permanent resident, of its system of adoption. All three problems are delicate. Fortunately, it is possible to address them separately. Detection We can define two general categories of detection policy: nominal and structural. In both cases the problem is to detect a mismatch between two versions of an object’s generating class: the version used by the system that stored the object, and the version used by the system which retrieves it. In the nominal approach, each class version is identified by a version name. This assumes some kind of registration mechanism, which may have two variants: • If you are using a configuration management system, you can register each new version of the class and get a version name in return (or specify the version name yourself)
$31.3 SCHEMA EVOLUTION 1043 More automatic schemes are possible,similar to the automatic identification facility of Microsoft's OLE 2,or the techniques used to assign "dynamic IP addresses"to computers on the Internet (for example a laptop that you plug in temporarily into a new network).These techniques are based on random number assignments,with numbers so large as to make the likelihood of a clash infinitesimal. Either solution requires some kind of central registry.If you want to avoid the resulting hassle,you will have to rely on the structural approach.The idea here is to associate with each class version a class descriptor deduced from the actual structure of the class,as defined by the class declaration,and to make sure that whenever a persistent mechanism stores objects it also stores the associated class descriptors.(Of course if you store many instances ofa class you will only need to store one copy of the class descriptor. Then the detection mechanism is simple:just compare the class descriptor ofeach retrieved object with the new class descriptor.If they are different,you have an object mismatch. What goes into a class descriptor?There is some flexibility;the answer is a tradeoff between efficiency and reliability.For efficiency,you will not want to waste too much space for keeping class information in the stored structure,or too much time for comparing descriptors at retrieval time;but for reliability you will want to minimize the risk of missing an object mismatch-of treating a retrieved object as up-to-date if it is in fact obsolete.Here are various possible strategies: CI.At one extreme,the class descriptor could just be the class name.This is generally insufficient:if the generator of an object in the storing system has the same name as a class in the retrieving system,we will accept the object even though the two classes may be totally incompatible.Trouble will inevitably follow. C2.At the other extreme,we might use as class descriptor the entire class text-perhaps not as a string but in an appropriate internal form(abstract syntax tree).This is clearly the worst solution for efficiency,both in space occupation and in descriptor comparison time.But it may not even be right for reliability,since some class changes are harmless.Assume for example the new class text has added a routine, but has not changed any attribute or invariant clause.Then nothing bad can happen if we consider a retrieved object up-to-date;but if we detect an object mismatch we may cause some unwarranted trouble(such as an exception)in the retrieving system. C3.A more realistic approach is to make the class descriptor include the class name and the list of its attributes,each characterized by its name and its type.As compared to the nominal approach,there is still the risk that two completely different classes might have both the same name and the same attributes,but (unlike in case Cl) such chance clashes are extremely unlikely to happen in practice. C4.A variation on C3 would include not just the attribute list but also the whole class invariant.With the invariant you should be assured that the addition or removal of a routine,which will not yield a detected object mismatch,is harmless,since if it changed the semantics of the class it would affect the invariant. C3 is the minimum reasonable policy,and in usual cases seems a good tradeoff,at least to start
§31.3 SCHEMA EVOLUTION 1043 • More automatic schemes are possible, similar to the automatic identification facility of Microsoft’s OLE 2, or the techniques used to assign “dynamic IP addresses” to computers on the Internet (for example a laptop that you plug in temporarily into a new network). These techniques are based on random number assignments, with numbers so large as to make the likelihood of a clash infinitesimal. Either solution requires some kind of central registry. If you want to avoid the resulting hassle, you will have to rely on the structural approach. The idea here is to associate with each class version a class descriptor deduced from the actual structure of the class, as defined by the class declaration, and to make sure that whenever a persistent mechanism stores objects it also stores the associated class descriptors. (Of course if you store many instances of a class you will only need to store one copy of the class descriptor.) Then the detection mechanism is simple: just compare the class descriptor of each retrieved object with the new class descriptor. If they are different, you have an object mismatch. What goes into a class descriptor? There is some flexibility; the answer is a tradeoff between efficiency and reliability. For efficiency, you will not want to waste too much space for keeping class information in the stored structure, or too much time for comparing descriptors at retrieval time; but for reliability you will want to minimize the risk of missing an object mismatch — of treating a retrieved object as up-to-date if it is in fact obsolete. Here are various possible strategies: C1 • At one extreme, the class descriptor could just be the class name. This is generally insufficient: if the generator of an object in the storing system has the same name as a class in the retrieving system, we will accept the object even though the two classes may be totally incompatible. Trouble will inevitably follow. C2 • At the other extreme, we might use as class descriptor the entire class text — perhaps not as a string but in an appropriate internal form (abstract syntax tree). This is clearly the worst solution for efficiency, both in space occupation and in descriptor comparison time. But it may not even be right for reliability, since some class changes are harmless. Assume for example the new class text has added a routine, but has not changed any attribute or invariant clause. Then nothing bad can happen if we consider a retrieved object up-to-date; but if we detect an object mismatch we may cause some unwarranted trouble (such as an exception) in the retrieving system. C3 • A more realistic approach is to make the class descriptor include the class name and the list of its attributes, each characterized by its name and its type. As compared to the nominal approach, there is still the risk that two completely different classes might have both the same name and the same attributes, but (unlike in case C1) such chance clashes are extremely unlikely to happen in practice. C4 • A variation on C3 would include not just the attribute list but also the whole class invariant. With the invariant you should be assured that the addition or removal of a routine, which will not yield a detected object mismatch, is harmless, since if it changed the semantics of the class it would affect the invariant. C3 is the minimum reasonable policy, and in usual cases seems a good tradeoff, at least to start
1044 OBJECT PERSISTENCE AND DATABASES $31.3 Notification What should happen when the detection mechanism,nominal or structural,has caught an object mismatch? We want the retrieving system to know,so that it will be able to take the appropriate correction actions.A library mechanism will address the problem.Class GENERAL (ancestor of all classes)must include a procedure correct mismatch is correct in this proce- dure name is not an do adjective but a verb, ...See full version below .. asin“Correct this mismatch,fast!".See end “Grammatical cate- gories".page 881. with the rule that any detection of an object mismatch will cause a call to correct mismatch on the temporarily retrieved version of the object.Any class can redefine the default version of correct mismatch;like a creation procedure,and like any redefinition of the default exception handling procedure default rescue,any redefinition of correct mismatch must ensure the invariant of the class. What should the default version of correct mismatch do?It may be tempting,in the name of unobtrusiveness,to give it an empty body.But this is not appropriate,since it would mean that by default object retrieval mismatches will be ignored-leading to all kinds of possible abnormal behavior.The better global default is to raise an exception: correct mismatch is --Handle object retrieval mismatch. do raise mismatch exception end where the procedure called in the body does what its name suggests.It might cause some "THE GLOBAL unexpected exceptions,but this is better than letting mismatches go through undetected. INHERITANCE A project that wants to override this default behavior,for example to execute a null STRUCTURE”, page 580. instruction rather than raise an exception,can always redefine correct mismatch,at its own risk,in class ANY.(As you will remember,developer-defined classes inherit from GENERAL not directly but through ANY,which a project or installation can customize.) For more flexibility,there is also a feature mismatch information of type ANY,defined as a once function,and a procedure set_mismatch_information (info:ANY)which resets its value.This makes it possible to provide correct_mismatch with more information,for example about the various preceding versions of a class. If you do expect object mismatches for a certain class,you will not want the default exception behavior for that class:instead you will redefine correct mismatch so as to update the retrieved object.This is our last task:correction
1044 OBJECT PERSISTENCE AND DATABASES §31.3 Notification What should happen when the detection mechanism, nominal or structural, has caught an object mismatch? We want the retrieving system to know, so that it will be able to take the appropriate correction actions. A library mechanism will address the problem. Class GENERAL (ancestor of all classes) must include a procedure correct_mismatch is do …See full version below … end with the rule that any detection of an object mismatch will cause a call to correct_mismatch on the temporarily retrieved version of the object. Any class can redefine the default version of correct_mismatch; like a creation procedure, and like any redefinition of the default exception handling procedure default_rescue, any redefinition of correct_ mismatch must ensure the invariant of the class. What should the default version of correct_mismatch do? It may be tempting, in the name of unobtrusiveness, to give it an empty body. But this is not appropriate, since it would mean that by default object retrieval mismatches will be ignored — leading to all kinds of possible abnormal behavior. The better global default is to raise an exception: correct_mismatch is -- Handle object retrieval mismatch. do raise_mismatch_exception end where the procedure called in the body does what its name suggests. It might cause some unexpected exceptions, but this is better than letting mismatches go through undetected. A project that wants to override this default behavior, for example to execute a null instruction rather than raise an exception, can always redefine correct_mismatch, at its own risk, in class ANY. (As you will remember, developer-defined classes inherit from GENERAL not directly but through ANY, which a project or installation can customize.) For more flexibility, there is also a feature mismatch_information of type ANY, defined as a once function, and a procedure set_mismatch_information (info: ANY) which resets its value. This makes it possible to provide correct_mismatch with more information, for example about the various preceding versions of a class. If you do expect object mismatches for a certain class, you will not want the default exception behavior for that class: instead you will redefine correct_mismatch so as to update the retrieved object. This is our last task: correction. correct in this procedure name is not an adjective but a verb, as in “Correct this mismatch, fast!”. See “Grammatical categories”, page 881. “THE GLOBAL INHERITANCE STRUCTURE”, page 580
$31.3 SCHEMA EVOLUTION 1045 Correction How do we correct a object that has been found,upon retrieval,to cause a mismatch?The answer requires a careful analysis,and a more sophisticated approach than has usually been implemented by existing systems or proposed in the literature. The precise situation is this:the retrieval mechanism (through feature retrieved of class STORABLE,a database operation,or any other available primitive)has created a new object in the retrieving system,deduced from a stored object with the same generating class;but it has also detected a mismatch.The new object is in a temporary state and may be inconsistent;it may for example have lost a field which was present in the stored object,or gained a field not present in the original.Think of it as a foreigner without a visa. -The attribute for this field Object 0.0 was not in the stored mismatch version,the field has been The attributes for these two initialized to the default fields have not changed from value for the attribute's type. the stored object's generating class to the new version. The stored object had a field here,but the new version of the class has removed the corresponding attribute; so the field has been lost. See“The role of cre. Such an object state is similar to the intermediate state of an object being created- ation procedures”, outside of any persistence consideration-by a creation instruction !x.make(...),just page 372. after the object's memory cell has been allocated and initialized to default values,but just before make has been called.At that stage the object has all the required components but is not yet ready for acceptance by the community since it may have inconsistent values in some of its fields;it is,as we saw,the official purpose of a creation procedure make to override default initializations as may be needed to ensure the invariant. Let us assume for simplicity that the detection technique is structural and based on attributes (that is to say,policy C3 as defined earlier),although the discussion will transpose to the other solutions,nominal or structural.The mismatch is a consequence of a change in the attribute properties of the class.We may reduce it to a combination of any number of attribute additions and attribute removals.(If a class change is the replacement of the type of an attribute,we can consider it as a removal followed by an addition.)The figure above shows one addition and one removal. Attribute removal does not raise any apparent difficulty:if the new class does not include a certain attribute present in the old class,the corresponding object fields are not needed any more and we may simply discard them.In fact procedure correct mismatch does not need to do anything for such fields,since the retrieval mechanism,when creating a tentative instance of the new class,will have discarded them;the figure shows this for the bottom field-rather,non-field-of the illustrated object
§31.3 SCHEMA EVOLUTION 1045 Correction How do we correct a object that has been found, upon retrieval, to cause a mismatch? The answer requires a careful analysis, and a more sophisticated approach than has usually been implemented by existing systems or proposed in the literature. The precise situation is this: the retrieval mechanism (through feature retrieved of class STORABLE, a database operation, or any other available primitive) has created a new object in the retrieving system, deduced from a stored object with the same generating class; but it has also detected a mismatch. The new object is in a temporary state and may be inconsistent; it may for example have lost a field which was present in the stored object, or gained a field not present in the original. Think of it as a foreigner without a visa. Such an object state is similar to the intermediate state of an object being created — outside of any persistence consideration — by a creation instruction !! x ● make (…), just after the object’s memory cell has been allocated and initialized to default values, but just before make has been called. At that stage the object has all the required components but is not yet ready for acceptance by the community since it may have inconsistent values in some of its fields; it is, as we saw, the official purpose of a creation procedure make to override default initializations as may be needed to ensure the invariant. Let us assume for simplicity that the detection technique is structural and based on attributes (that is to say, policy C3 as defined earlier), although the discussion will transpose to the other solutions, nominal or structural. The mismatch is a consequence of a change in the attribute properties of the class. We may reduce it to a combination of any number of attribute additions and attribute removals. (If a class change is the replacement of the type of an attribute, we can consider it as a removal followed by an addition.) The figure above shows one addition and one removal. Attribute removal does not raise any apparent difficulty: if the new class does not include a certain attribute present in the old class, the corresponding object fields are not needed any more and we may simply discard them. In fact procedure correct_mismatch does not need to do anything for such fields, since the retrieval mechanism, when creating a tentative instance of the new class, will have discarded them; the figure shows this for the bottom field — rather, non-field — of the illustrated object. Object mismatch The attribute for this field was not in the stored version; the field has been initialized to the default value for the attribute’s type. The stored object had a field here, but the new version of the class has removed the corresponding attribute; so the field has been lost. The attributes for these two fields have not changed from the stored object’s generating class to the new version. 0.0 See “The role of creation procedures”, page 372
1046 OBJECT PERSISTENCE AND DATABASES $31.3 We might of course be a bit more concerned about the discarded fields;what if they were really needed,so that the object will not make sense without them?This is where having a more elaborate detection policy,such as structural policy C4 which takes the invariant into account,would be preferable. The more delicate case is when the new class has added an attribute,which yields a See"UniformAccess" new field in the retrieved objects,as illustrated by the top field of the object in the page 55,and "Defini- preceding figure.What do we do with such a field?We must initialize it somehow.In the tion and example” page 364. systems I have seen offering some support for schema evolution and object conversion, the solution is to use a conventional default as initialization value (the usual choices:zero for numbers,empty for strings).But,as we know from earlier discussions of similar problems-arising for example in the context of inheritance-this may be very wrong! Our standard example was a class ACCOUNT with attributes deposits list and withdrawals list;assume that a new version adds an attribute balance and a system using this new version attempts to retrieve an instance created from the previous version. deposits list Retrieving an S900 S850 S250 account object Old fields withdrawals list 300 700 (What is wrong New field (initialized to with this 0.0 balance default value of its type) picture?) The purpose of adding the balance attribute is clear:instead of having to recompute an account's balance on demand we keep it in the object and update it whenever needed. The new class invariant reflects this through a clause of the form balance deposits list.total-withdrawals list.total But if we apply the default initialization to a retrieved object's balance field,we will get a badly inconsistent result,whose balance field does not agree with the record of deposits and withdrawals.On the above figure,balance is zero as a result of the default initialization;to agree with the deposits and withdrawals shown,it should be 1000 dollars Hence the importance ofhaving the correct mismatch mechanism.In such a case the class will simply redefine the procedure as correct mismatch is --Handle object retrieval mismatch by correctly setting up balance do balance:=deposits list.total-withdrawals list.total end If the author of the new class has not planned for this case,the default version of correct mismatch will raise an exception,causing the application to terminate abnormally unless a retry(providing another recovery possibility)handles it.This is the right outcome, since continuing execution could destroy the integrity of the execution's object structure- and,worse yet,of the persistent object structure,for example a database.In the earlier metaphor,we will reject the object unless we can assign it a proper immigration status
1046 OBJECT PERSISTENCE AND DATABASES §31.3 We might of course be a bit more concerned about the discarded fields; what if they were really needed, so that the object will not make sense without them? This is where having a more elaborate detection policy, such as structural policy C4 which takes the invariant into account, would be preferable. The more delicate case is when the new class has added an attribute, which yields a new field in the retrieved objects, as illustrated by the top field of the object in the preceding figure. What do we do with such a field? We must initialize it somehow. In the systems I have seen offering some support for schema evolution and object conversion, the solution is to use a conventional default as initialization value (the usual choices: zero for numbers, empty for strings). But, as we know from earlier discussions of similar problems — arising for example in the context of inheritance — this may be very wrong! Our standard example was a class ACCOUNT with attributes deposits_list and withdrawals_list; assume that a new version adds an attribute balance and a system using this new version attempts to retrieve an instance created from the previous version. The purpose of adding the balance attribute is clear: instead of having to recompute an account’s balance on demand we keep it in the object and update it whenever needed. The new class invariant reflects this through a clause of the form balance = deposits_list ● total – withdrawals_list ● total But if we apply the default initialization to a retrieved object’s balance field, we will get a badly inconsistent result, whose balance field does not agree with the record of deposits and withdrawals. On the above figure, balance is zero as a result of the default initialization; to agree with the deposits and withdrawals shown, it should be 1000 dollars. Hence the importance of having the correct_mismatch mechanism. In such a case the class will simply redefine the procedure as correct_mismatch is -- Handle object retrieval mismatch by correctly setting up balance do balance := deposits_list ● total – withdrawals_list ● total end If the author of the new class has not planned for this case, the default version of correct_mismatch will raise an exception, causing the application to terminate abnormally unless a retry (providing another recovery possibility) handles it. This is the right outcome, since continuing execution could destroy the integrity of the execution’s object structure — and, worse yet, of the persistent object structure, for example a database. In the earlier metaphor, we will reject the object unless we can assign it a proper immigration status. See “Uniform Access”, page 55, and “Definition and example”, page 364. Retrieving an account object (What is wrong with this picture?) 0.0 Old fields New field (initialized to default value of its type) withdrawals_list deposits_list balance $900 $850 $250 $300 $700
