Validation

From Qubit Toolkit
Jump to: navigation, search
Design This page proposes a new feature and reviews design options
Development This page describes a feature that is currently in development
Documentation This page documents an existing feature

There are a number of places in the application where we need to validate data entry to ensure that the data,

  1. is compliant with the professional standards that the application supports (e.g. mandatory fields)
  2. meets any technical requirements to ensure the application runs smoothly (e.g. minimum number of characters)

Contents

[edit] Requirements

Each of the four ICA standards supported in Qubit has a set of mandatory elements that should be enforced on their respective data-entry forms,

[edit] ICA-ISAD(G)

  • Mandatory elements,
    • Reference code (completed)
    • Title (completed)
    • Creator (completed)
    • Date(s) (completed) (see comments below)
    • Extent of the unit of description (completed)
    • Level of description (completed) (see comments below)

[edit] Date(s)

The software should control the compliance of the dates of the low-level units with the dates of the high-level units

User:Jesus: Iterate over higher unit of descriptions checking if dates intervals match.

User:Jablko http://logs.jabber.org/qubit@conference.jabber.org/2009-11-27.html#17:53:02

User:Evelyn Have to make sure that date range for creation at higher level doesn't prevent other kinds of dates being added at lower levels. For example, the creation date for a fonds could be 1950-1960, but an item within the fonds could have been published in 1980.

User:Jesus Resolved in r4376 with a specific validator. This may could be resolved more elegant in the future. Btw, I wasn't sure if a creation/... date is required, or just optional.

[edit] Level of description

ica-atom:RS-1#3.1.4

The software should control whether the links between hierarchical levels are logical (for example, the software should warn the user if he links a fonds-level description as a child-level unit of a file-level description)

Jesús: I guess that we should add a weight field (int) into q_term table to know if a Fond is a level higher than a Serie. We can describe these priorities for existing values in a class, but I don't know how to do it with new levels of description introduced by users.

User:Jablko Maybe to get us started, we could just check the relationships from the standard? Maybe we could use the sfValidatorBlacklist from the sfFromExtraPlugin to check that the level of description is not an illogical value? e.g.


// Check that object has a parent level of description, i.e. a parent which is not the root
if (isset($this->informationObject->parent->parent))
{
  // Unset illogical level of description values based on the parent level of description
  $forbiddenValues = array('item', 'file', 'sub-series', 'series', 'sub-fonds', 'fonds');
  foreach ($forbiddenValues as $key => $value)
  {
    if ($value == strtolower($this->informationObject->parent->levelOfDescription))
    {
      break;
    }
 
    unset($forbiddenValues[$key]);
  }
 
  $validatorSchema->levelOfDescription = new sfValidatorBlacklist('forbidden_values' => $forbiddenValues, 'required' => true);
}

User: Peter The Level of Description hierarchy needs to stay flexible, allowing users to modify it for institutional or national practices. Therefore, we can't hardcode a Level hierarchy. The application will ship with our current default set of Levels which the user can then revise via the standard Term edit screens. This order should be used to validate the Level of Description hierarchy. In an earlier release we had a 'sort' column for Terms. We got rid of that in favour of (eventually) using the Nested set 'lft' & 'rgt' values for sorting amongst siblings. Now that the Term object is using a nested set we have to implement this. The ideal way to implement that is to allow drag-and-drop in the hierarchy tree (like we currently allow to change parentId) but then specific to the siblings in the branch. Then we can use the 'lft' value of the siblings as the hierarchy sort order and we can set a validation rule that ensures a particular information object's term->lft value is not less than any of its ancestors.

  • To implement this we will need the "moveToNextSiblingOf(...)" and "moveToPrevSiblingOf(...)" methods in the Qubit NestedSet. However, adding that method will likely require some significant development and testing time. This raises the question of how much additional development time we want to invest in our own custom Nested Set implementation in light of the fact that these are now more robustly supported in both Propel and Doctrine. The future of our ORM is a discussion we will need to revisit after our current dev deadlines.

Regardless, the ability to sort the Level of Description taxonomy and then enforce those ordered hierarchy levels as a Validation rule, will be a high priority requirement for 1.1. If we can't provide that via NestedSet ordering and treeview drag-n-drop then we may have to look at an alternative work-around (i.e. adding a sort/weight property for the Level Of Description taxonomy)

In the meanwhile, I think we should just look for and enforce the Levels of Description and their respective order as specifically mentioned as examples in rule 1.3.4 of ISAD(G). Additionally, we include 'collection' as a top-level alternative to 'fonds' as this is supported by ISAD. "It is assumed that the same rules used to describe a fonds and its parts may be applied to the description of a collection" (p8)

We can simply ignore any Levels that may have been custom added by users in our validation checks for now. So for the ISAD Validation rule we just make sure that these rules are respected:

  • fonds cannot be child of collection
  • collection cannot be child of fonds
  • sub-fonds cannot be child of collection
  • all levels in both hierarchy option 1 or option 2 are optional. The only logic we are validating is that an ancestor level is not added as a descendant (e.g. a fonds cannot be a child of a file, a series cannot be a child of an item, a collection cannot be a child of a series, etc)
  • hierarchy option 1
  1. Fonds
  2. Sub-fonds
  3. Series
  4. Sub-series
  5. File
  6. Item
  • hierarchy option 2
  1. Collection
  2. Series
  3. Sub-series
  4. File
  5. Item

User:Jesus In r4309 I implemented the validator as you described. However, I wasn't sure what is more correct when checking that "an ancestor level is not added as a descendant". Do you think this should be checked against the parent level of description or against all the values collected from all the ancestors? r4309 checks it in the first way but could be improvable easily.

see also Googlegroups discussion on this issue

[edit] ICA-ISAAR(CPF)

  • Mandatory elements,
    • Type of entity (completed)
    • Authorized form(s) of name (completed)
    • Dates of existence (completed)
    • Authority record identifier (completed)
  • Other requirements,
    • The "type" option for the "Other names" row should be filtered according to the ISAAR entity type. For example, values such as maiden name, first name, nickname, etc. should not appear when you create/edit an authority record that is a 'corporate body' entity type. (finally, this feature is not required, see issue 1180 or discussion thread @qubit-dev.

[edit] ICA-ISDIAH

  • Mandatory elements,
    • Identifier (completed)
    • Authorised form(s) of name (completed)
    • Location and address(es) (completed)

[edit] Location and address(es)

ica-atom:RS-3#4

Current rules we are checking,

  1. Repository has a primary contact
  2. Primary contact has *any* of,
    • city
    • country
    • postal code
    • region
    • street address

Should we check any additional rules? alternative rules?

  •  ???

[edit] ICA-ISDF

  • Mandatory elements,
    • Type (completed)
    • Authorised name of the function/activity (completed)
    • Function/activity description identifier (completed)

[edit] RAD

Secs. 1.0D1 to 1.0D3 of RAD specify required elements for the following levels of description: fonds, collection, series, file, item. The required elements common to all these levels of description are as follows:

  • Title proper (completed)
  • Class of materials specific details (completed)
  • Dates of creation (at item level, may be replaced by date of publication, distribution etc.) (completed) (see details below)
  • Extent of descriptive unit (in the form appears as "Physical description") (completed)
  • Scope and content (completed)
  • Notes (in progress)

Extra required elements for specific levels of description are as follows:

  • Fonds, collection, series (assuming that the fonds, collection or series is the top level of description)
    • Administrative history/Biographical sketch/History (in progress)
    • Custodial history (completed)
  • File
    • No extra elements
  • Item (assuming that it is a publication) (in progress)
    • Edition statement
    • Standard number
    • User:Peter As Evelyn points out, these are only valid if the item type is a publication. Therefore, this validation rule should only be enforced if the item-level QubitInformationObject is linked to a QubitEvent where typeId == QubitTerm::PUBLICATION_ID

[edit] Class of materials specific details

  • User:Sevein This is really an area of five fields. For the time being, I made those five fields as mandatory ones. Should I keep this behavior or just to makes the validator fail when all of them were not completed?
  • User:Peter This is one is more complicated. They are only applicable if a matching General Material Designation is chosen. i.e. if 'Architectural drawing' is chosen then 'Statement of scale (architectural)' is required:
    • Architectural drawing: Statement of scale (architectural)
    • Cartographic material: Statement of scale (cartographic), Statement of coordinates (cartographic), Statement of scale (architectural)
    • Philatelic material: Issuing jurisdiction and denomination (philatelic)
  • User:Evelyn The problem with this approach is that GMD is not a required field. So if the user didn't select a GMD, the Class of material specific details warnings wouldn't appear. Also, if the fonds is very diverse, the user may select “Multiple media” as the GMD, in which case media-specific Class of material specific detail warnings wouldn't appear.

[edit] Notes

  • User:Evelyn It's not clear to me how Notes can be a required element.

[edit] Dates of creation

  • User:Jesus It is already implemented but, at item level, what event types should be exactly checked? When a event row is not considered a valid date? start_date/end_date/show_date fields empty?

[edit] Administrative history/Biographical sketch/History

This value belongs to creators (actors). When should the validator fails?

  • User:Jesus ?? In case of existence of creators with this field empty. ?? In case of inexistence of creators

[edit] Other observations

  • User:Evelyn Implementing full validation for RAD would require different edit templates for different levels of description and possibly for different types of records. A workaround may be to implement full rules for common elements, and for specific elements add notes to the field labels in the edit screen such as "Required element at highest level of description", "Required element for item-level description" etc.
  • User:Peter We need some further communication with LAC and the CCAD committee responsible for RAD as there are a number of incongruences. I will take care of that in early January as part of our response to the LAC review of RAD in ICA-AtoM 1.0.8.
  • User:Jesus If level of description value is different of 'Fonds', 'Collection', 'Series', 'File' or 'Item', any validation will be applied. Is this right?

[edit] Relationships

  • User:Evelyn In ISDF, the user shouldn't be able to create a relationship between a Function and itself. See issue issue #1194. This may also be happening in Terms (eg selecting a BT that is the same as the current term) and ISAAR records (selecting the current record's name as the Name of related entity) but I can't tell because the auto-complete lists in those edit screens are broken - see issue #1195 and #1196.

[edit] User management

  • Basic info
    • Username - required field (issue 1244) (completed)
    • Email - required field (issue 1244) (completed)
    • Password - required field (issue 1244) (completed) and confirm value (completed)
  • Groups and permissions
    • Groups - (?)
    • Permissions - (?)

[edit] Generic requirements

  • Search wildcard requires minimum of three characters (e.g. 'abc*'), otherwise Zend Lucene crashes, issue 985
  • Don't allow duplicate terms in the same taxonomy, users can inadvertently enter a new or duplicate term in subject or place access points by typing in the term and going on to enter data in other parts of the screen, then saving the record. The system does not force the user to choose a term from the drop-down list or warn them if they have entered a new or duplicate term, issue 1133
  • Add warning when user adds values to child-level objects that should be inherited from parent objects (i.e. actor name, repository). Warning could read, for example: "Repository is duplicate of inherited value and should be deleted." Or something like that. http://code.google.com/p/qubit-toolkit/issues/detail?id=1304

[edit] Design

Some places in the application assume that information objects have titles when they may not. The two options are to require titles, or to display something helpful if there is no title.

  • see issue 528
  • for 1.0.7 we decided to display 'Untitled' if a title was not entered

Drupal requires a title, so this may be a good approach. However the first place to require a title is the database schema, and because title is translatable, this is a bit awkward. Even if validation is done in the form, must a translator translate the title before making additional translations?

The other alternative is rendering something helpful. The link text comes from toString(), which already implements culture fallback. What is an example of helpful text that is returned when there is no title?

  • No title
  • Anonymous
  • No name
  • Nameless
  • Unnamed
  • Undefined
  • None
  •  ???

Implementing this makes us avoid overzealous validation, while forcing stricter use of toString(), which may be good. What is the advantage of not implementing this, and implementing validation instead?

Another thing which could be nice is a helper for generating links, so that links get titles set to something meaningful

http://yuilibrary.com/projects/yui2/ticket/2527672

I think using a helper is appropriate because it puts markup in the view layer, instead of the model layer. I'm reluctant to use the word "format" because it's used for i18n. I'm reluctant to use the word "show" because it's used for displaying the whole object. to_string is not appropriate because we are rendering HTML. "render" is a possibility. What I really want is the handle, the name, the identifier, or the title of the object.

I would call it the title, but I'm trying to enforce a separation between the metadata element and the presentation component. Should this be done on the client side? Is there a CSS selector for empty titles? Like if you could coax link_to() to render an empty <a/>, could the stylesheet put something inside? This is much more experimental.

render_title() is the best I can think of tonight. Title because we're not rendering the whole object, and render title because it is not the actual title, but a presentation version of it.

[edit] Automating required fields

The forms system should be able to tell which columns of the model are required, providing nicer error messages for free, http://groups.google.com/group/qubit-dev/browse_thread/thread/427019ab5ae394dd

[edit] Style

[edit] Show pages

Current desgin:

ValidationErrorMessageStyle 03.png

User:Evelyn Perhaps name of field could link to the field description in the standard, and the words "requires" and/or "mandatory" could link to the section of the standard which lists the mandatory elements? Examples:

[edit] Edit pages

When users submit invalid values ($this->form->isValid() is false), edit**Success template is displayed again and validation process adds the error messages into the form. Below is the way these validation error message are shown now. Should be improved? <li> shows more left-padding than needed and list icon doesn't look nice with that red background.

ValidationErrorMessageStyle 01.png

This one below could be implemented but some code from Symfony must be modified:

ValidationErrorMessageStyle 02.png

User:Jablko Is it possible to leave this up to individual themes?

Moreover, what about <fieldset> tag? When edit page is loaded, fieldsets are closed by default. What should happen when a fieldset contents a form field with validation error messages? Should be opened by client side (updating code which search for fieldset with 'collapsible' CSS class) or server side (checking fields and removing CSS 'collapsed' class)?

User:Jablko I think this will fix the problem - but please gimme a day to think on it, http://www.sfu.ca/~jdbates/tmp/qubit/200911161/patch

User:Evelyn I am thinking that maybe asterisks alone are enough. Asterisks in edit screen plus validation messages in view screen seems like plenty of validation to me.

User:Peter I agree with Evelyn. Asteriks on the edit templates & validation messages on the show templates (only for those users with 'edit' permission). See further discussion

Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox