Views: 14<\/p>
Hadley Wickham defines tidy data as follows: \nThis definition of tidy data facilitates data analysis and visualization and allows consistent data handling. It is important to note that tidy data imposes restrictions on the types of observations by specifying “each type of observational unit”. If we remove this restriction and just specify “observational units”,this means that various types of data can be handled in a consistent format, significantly enhancing the flexibility of data analysis.\n<\/p>\n<\/div>\n<\/div>\n<\/div>\n Hierarchical Tidy Data extends Wickham’s principles of tidy data to efficiently represent datasets with multiple hierarchies. This approach integrates data into a single tabular format while maintaining relationships between different levels of hierarchy. Traditionally, normalized multiple tables and JOIN operations were used to manipulate necessary information. In contrast, Hierarchical Tidy Data incorporates all necessary information into a single table, using rows to correspond to different observations, thereby eliminating data redundancy.<\/p>\n<\/div>\n<\/div>\n<\/div>\n The following table represents information about classes, students, subjects, and scores as hierarchical tidy data:<\/p>\n<\/div>\n NULL<\/p>\n<\/td>\n 1<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Alice<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Student Alice<\/p>\n<\/td>\n<\/tr>\n NULL<\/p>\n<\/td>\n 2<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Bob<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Student Bob<\/p>\n<\/td>\n<\/tr>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n 1<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Math<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Subject Math<\/p>\n<\/td>\n<\/tr>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n 2<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Science<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Subject Science<\/p>\n<\/td>\n<\/tr>\n 1<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Class 1<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Class 1<\/p>\n<\/td>\n<\/tr>\n 1<\/p>\n<\/td>\n 1<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Alice<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Alice is in Class 1<\/p>\n<\/td>\n<\/tr>\n 1<\/p>\n<\/td>\n 1<\/p>\n<\/td>\n 1<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n 90<\/p>\n<\/td>\n Test result of Alice’s score of Math is 90 in Class 1<\/p>\n<\/td>\n<\/tr>\n 1<\/p>\n<\/td>\n 1<\/p>\n<\/td>\n 2<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n 85<\/p>\n<\/td>\n Test result of Alice’s score of Science is 85 in Class 1<\/p>\n<\/td>\n<\/tr>\n 2<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Class 2<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Class 2<\/p>\n<\/td>\n<\/tr>\n 2<\/p>\n<\/td>\n 2<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Bob<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n Bob is in Class 2<\/p>\n<\/td>\n<\/tr>\n 2<\/p>\n<\/td>\n 2<\/p>\n<\/td>\n 1<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n 80<\/p>\n<\/td>\n Test result of Bob’s score of Math is 80 in Class 2<\/p>\n<\/td>\n<\/tr>\n 2<\/p>\n<\/td>\n 2<\/p>\n<\/td>\n 2<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n NULL<\/p>\n<\/td>\n 88<\/p>\n<\/td>\n Test result of Bob’s score of Science is 88 in Class 2<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n In this table, information at each level (class, student, subject, score) is integrated, with each observation represented as a row. This eliminates data redundancy and allows efficient management of hierarchical relationships.<\/p>\n<\/div>\n<\/div>\n<\/div>\n Class Level<\/strong>: Student Level<\/strong>: Subject Level<\/strong>: Score Level<\/strong>: Observation<\/strong>: Provides a human-readable description of the observation at each level, encapsulating the hierarchical relationship.<\/p>\n<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n To maintain the integrity of hierarchical tidy data, unique key constraints must be defined at each level of the hierarchy, allowing nulls to accommodate the hierarchical nature. These constraints ensure that the combination of Class Level<\/strong>:<\/p>\n Unique Key<\/strong>: Allows nulls in Student Level<\/strong>:<\/p>\n Unique Key<\/strong>: Allows nulls in Subject Level<\/strong>:<\/p>\n Unique Key<\/strong>: Allows nulls in Student in Class Level<\/strong>:<\/p>\n Unique Key<\/strong>: Composition of Allows nulls in Score Level<\/strong>:<\/p>\n Unique Key<\/strong>: Composition of Uniquely identifies each test score for a subject for each student within their respective class.<\/p>\n<\/li>\n<\/ul>\n<\/div>\n<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div>\n Class Level<\/strong>: Student in Class Level<\/strong>: Score Level<\/strong>: By allowing nulls in the unique key components, we ensure that each level’s identifiers uniquely identify the corresponding entity while maintaining the hierarchical relationships and avoiding redundancy.<\/p>\n<\/div>\n Dimensions defined in the XBRL definition linkbase can represent this structure and generate hierarchical tidy data in a single CSV file using xBRL-CSV. This approach eliminates the need for traditional normalized multiple tables and joins in a relational database. All necessary data is integrated into one file, making it easy to handle without a database.<\/p>\n<\/div>\n Additionally, historical data, such as records from 10 years ago, can be accessed without needing a database. This frees users from regular version updates and maintenance. Since all data, including master data, is stored as is, past data can be used without needing to restore and join past master data. This ensures that all information, including historical data, is consistently maintained, facilitating data analysis and reporting.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n1. Each variable forms a column<\/strong>: Each variable in a dataset is placed in its own column.
\n2. Each observation forms a row<\/strong>: Each observation is placed in its own row.
\n3. Each type of observational unit forms a table<\/strong>: Each cell contains a single value.<\/p>\n<\/div>\n2. Extension to Hierarchical Tidy Data<\/h2>\n
3. Example<\/h2>\n
\n
\n \nClassID<\/th>\n Student ID<\/th>\n Subject ID<\/th>\n Class Name<\/th>\n Student Name<\/th>\n Subject<\/th>\n Score<\/th>\n Observation<\/th>\n<\/tr>\n<\/thead>\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n 4. Explanation of Hierarchical Levels<\/h2>\n
\n
ClassID<\/code> and Class Name<\/code> represent the details of each class.<\/p>\n<\/li>\nStudent ID<\/code> represents the details of each student.<\/p>\n<\/li>\nSubject<\/code> represents the details of each subject.<\/p>\n<\/li>\nScore<\/code> represents the test scores for a subject for each student within their respective class.<\/p>\n<\/li>\n5. Unique Key Constraints Allowing Nulls<\/h2>\n
ClassID<\/code>, Student ID<\/code>, and Subject ID<\/code> uniquely identifies each observation.<\/p>\n<\/div>\n\n
\n
ClassID<\/code><\/p>\n<\/li>\nStudent ID<\/code> and Subject ID<\/code> to uniquely identify each class.<\/p>\n<\/li>\n<\/ul>\n<\/div>\n<\/li>\n\n
Student ID<\/code><\/p>\n<\/li>\nClassID<\/code> and Subject ID<\/code> to uniquely identify each student.<\/p>\n<\/li>\n<\/ul>\n<\/div>\n<\/li>\n\n
Subject ID<\/code><\/p>\n<\/li>\nClassID<\/code> and Student ID<\/code> to uniquely identify each subject.<\/p>\n<\/li>\n<\/ul>\n<\/div>\n<\/li>\n\n
ClassID<\/code> and Student ID<\/code><\/p>\n<\/li>\nSubject ID<\/code> to uniquely identifies each student within their respective class.<\/p>\n<\/li>\n<\/ul>\n<\/div>\n<\/li>\n\n
ClassID<\/code>, Student ID<\/code>, and Subject ID<\/code><\/p>\n<\/li>\n6. Example Explanation<\/h2>\n
\n
Class 1<\/code> with ClassID = 1<\/code> and Class 2<\/code> with ClassID = 2<\/code> are uniquely identified by their ClassID<\/code>.<\/p>\n<\/li>\nAlice<\/code> with Student ID = 1<\/code> in Class 1<\/code>\u3000with ClassID = 1<\/code> and Bob<\/code> with Student ID = 2<\/code> in Class 2<\/code> with ClassID = 2<\/code> are uniquely identified by the combination of ClassID<\/code> and Student ID<\/code>.<\/p>\n<\/li>\nAlice’s<\/code> score of 90<\/code> in Math<\/code> for Class 1<\/code> is uniquely identified by the combination of ClassID<\/code>, Student ID<\/code>, Subject ID<\/code>.<\/p>\n<\/li>\n<\/ul>\n<\/div>\n