UDiscoverIt Tutorial/Qualification Overview
This tutorial is intended to demonstrate the key concepts to be used in the UDiscoverIt HITs and to test your understanding of these concepts. You may repeat the test if needed to reach the required score.
Estimated Tutorial Time: 20-30 minutes. Estimated HIT time: 30-60 seconds per HIT.
The task to be performed in the HITs covered by this tutorial will be:
The tutorial begins with the criteria for what features constitute a pattern, what can't be part of a pattern, and what features obscure the details of a pattern. This is followed by criteria for choosing which pattern to describe if you see more than one, and which lines/blobs to mark.
The tutorial will be broken into sections introducing groups of related concepts, with questions at the end of each section.
The examples (and related HITs) in this tutorial sometimes include numerical labels for features in the images (these are vertical lines/blobs, and horizontal row gridlines). If the labels are difficult to read, they are numbered sequentially, almost always beginning with 1. The questions are designed so that mistakes are unlikely due to mis-reading the numbers. In the tutorial, you can zoom when necessary by clicking the images.
In the tutorial, you will be asked to identify features like vertical lines/blobs or horizontal gridlines by numerical ID. However, in the real HITs you will identify vertical lines/blobs by clicking on them to toggle selection.
The colors of the vertical lines/blobs can be thought of in terms of brightness or intensity. The relative values are given by the following image. Low values are on the left (dark blue), and high values are on the right (red, white):
HIT Requirements and Helpful Skills
The tutorial is expected to take around 20-30 minutes to complete, and individual HITs are expected to take around 30-60 seconds each. The qualification test is graded automatically and you will be able to work on HITs immediately if you meet the minimum score indicated on the HITs. You may take the test multiple times if needed.
Academic Research Purpose
Responses for the HITs associated with this tutorial/qualification will be collected as data to be used for academic research purposes. Please indicate whether you accept this. If not, you can stop here (answering no will result in a failing score on the test).
Relationships Allowed in Patterns
The following are examples of the types of relationships that are likely to occur with patterns, paired with examples of relationships that can't be part of a pattern. Explanations are given for each. Typically, when you see many vertical lines/blobs that all have an allowed relationship, it is very likely to be a pattern. But when any pair show one of the incorrect relationships, they are almost certainly not a pattern.
|Incorrect vs. Permitted Pattern Relationship Examples||Explanation|
|Lines/blobs in the same pattern can have different widths, but not different vertical lengths.|
|Lines/blobs in the same pattern must have their tops and bottoms aligned. Alignment is determined by the horizontal gridlines that are crossed, so the lines/blobs in the correct example are aligned even though the middle one might appear shorter.|
|Lines/blobs in the same pattern can have different overall intensities and the intensities can change along their lengths, but they must change the same way.|
|Lines/blobs in the same pattern can shift diagonally, but they will almost always shift together. Note: the diagonal shift isn't necessarily parallel- it is expected to be slightly steeper for vertical lines/blobs farther to the right in the image. Shifts that don't match should make you much less confident in a pattern, however two lines with different shifts is not sufficient to say they are definitely in different patterns.|
|Lines/blobs in the same pattern can change widths along their lengths, but they must change together.|
Question: Properties of Patterns
You see a group of five vertical lines/blobs with the properties described below. Which description most likely refers to a single clear pattern?
Relationships that Obscure Patterns
The following are examples of the types of relationships that make it difficult to identify a pattern, or make it difficult to identify specifically which lines/blobs are part of the pattern. However, they might still provide useful context for more clear lines/blobs.
Generally, you should be less certain of any pattern that relies on lines/blobs with these relationships. Also, when choosing lines/blobs to mark, it is best to avoid marking lines/blobs with these relationships, unless all of the other options are more obscured.
In some cases, it would be possible to infer a pattern from lines/blobs with these relationships, but it will typically require too much effort. It's better to spend the time on easier ones.
|Explanation||Obscured Pattern Relationships Examples||Explanation|
|A single vertical line/blob generally will not provide enough information to identify a pattern on its own.||Vertical lines/blobs that only intersect a single horizontal gridline generally do not provide enough information to identify a pattern, the way they change is not visible. Note: there will usually be other images in the series providing more information about such patterns.|
|It is difficult to see the extent and changes in very faint lines, therefore it is best not to include these in pattern assignments, or to rely on them for pattern identification. A group of lines being faint or invisible does not make them a pattern.||Sometimes vertical lines/blobs will appear in almost exactly the same horizontal position. In some cases, they will have separate markers/labels as shown here (in which case you will be prevented from selecting them), and in other cases you must infer this from the visible changes and context. Do not mark these, because it is either impossible to choose the correct label, or difficult to make out their relationship to a pattern.|
|Lines/blobs from a pattern can be overlapping. In these cases, again it is best to look for more clearly separated examples because the variations are obscured.||Lines/blobs from different patterns can be overlapping, which again obscures the variation in an individual pattern.|
|Generally when vertical lines/blobs from the same pattern shift diagonally, they do so together. However, it is possible (though unlikely) for them to have different shift angles. This makes them unlikely to be a pattern, but is not sufficient to be certain they are in different patterns.||Though unlikely, steep shifts can introduce visual artifacts, where the blobs appear "stepped" or disconnected. There appear to be two pairs of blobs in this image, when there is "really" only one pair. The only way to tell the difference is context from other features, and it is not worth trying to decide this. In general, don't assume one way or the other (that they are a continuation of the same blobs, or new ones). When possible, only make selections that would be correct either way.|
Questions: Obscured Patterns
Why is it best to ignore, or discount the value of, vertical lines/blobs that overlap each other (check all that apply)?
You see in an image a group of 4 lines/blobs that are all too dark to make out details. They are:
Pattern Identification Examples
Simple Example: 1 pattern
This image has 3 vertical lines/blobs and 1 likely visible pattern. The two blobs on the upper-right are a pattern because:
It doesn’t matter that:
The bottom-left blob is separate because:
Simple Example: 2 patterns (blobs added to previous example)
This image has 5 vertical lines/blobs, and 2 patterns. The bottom-left blob forms a pattern with the two blobs on the bottom-right because:
It doesn’t matter that:
What is the most likely total number of patterns visible in this image, assuming each vertical line/blob is part of a pattern? (This is your best guess for the total number, not just clear patterns with un-obscured blobs.)
Find the clearest, most identifiable pattern in this image. We will call this the primary pattern. Vertical lines/blobs are labeled with numbers at the top and bottom of the image. Which numbered blobs are clearly and unambiguously part of the Primary Pattern?
Why do these form a pattern (check all that apply)?
Target Row Definition and Question
The Target Row is indicated by a series of red-and-white circles marking each vertical line/blob. The target row indicates the region of the image to look for patterns, in other words, we are looking for patterns that intersect this row. Note that observing patterns elsewhere in the image can still provide useful context. Refer to this image for the following question (click to zoom):
Which row is the target row in this image (row gridlines are numbered at the left side of the image, with #1 at the top and #7 at the bottom)?
Primary Pattern vs. Secondary Patterns (click to zoom)
If there are multiple patterns crossing the Target Row, you will choose the clearest pattern and call this the Primary Pattern; any remaining patterns are called Secondary Pattern(s). The main idea is to choose the pattern for which you can provide the most certain response. Use the following criteria to determine which is the best to select as the Primary Pattern:
If two or more patterns have similar clarity according to these criteria, just pick the one that makes it easiest for you to provide an answer with certainty.
This image appears to have 4 (or more) patterns, with a single vertical line/blob. Only 2 (or more) of them intersect the target row. We call the clearest pattern the Primary Pattern, and any other patterns are the Secondary Patterns. There can be zero, one or two Secondary Patterns, but we treat them all the same way and consider them together.
In this example image, we have chosen a Primary and Secondary Pattern, and we have marked the vertical lines/blobs that best identify each, and reasons are given below.
Primary Pattern (orange marks):
Secondary Pattern(s) (purple marks):
Questions: Overlapping Vertical Lines/Blobs
For the next 2 questions, refer to the "Obscure Patterns" section of the tutorial, and apply to this image (click to zoom):
Which vertical lines/blobs should we exclude from responses because the labels overlap, meaning they are too close to tell apart?
Which vertical lines/blobs might we exclude from responses because the blobs overlap even though the labels are separated, meaning it is hard to see if they are in a pattern?
It is questionable whether blob #1 should be marked as part of the Secondary Pattern(s). Why might we exclude it?
It is questionable whether blob #8 should be marked as part of the Primary Pattern. Why might we exclude it?
Example: Harder Images (click to zoom)
Some images will have lots of vertical lines/blobs that seem likely to be patterns, but with many overlapping or faint ones.
The Primary Pattern is the one for which we can choose the best set of vertical lines/blobs to define it with the most certainty.
In this case, it is best to have the Primary Pattern be the ones that extend to the bottom of the image, because all of the lines/blobs extending to the top of the image overlap with other lines/blobs.
The Secondary Pattern(s) would be those vertical lines/blobs that extend to the top of the image and therefore can’t be part of the Primary Image.
Note: we can't always tell how many patterns there are. In this image, there are at least 2 patterns, but it's possible there are more. We would leave many of the vertical lines/blobs unmarked in this image because they are likely to be part of the Primary or Secondary Patterns, but we can't be sure.
Questions: Harder Images
The next 2 questions refer to the same image as in the preceding example. Again, we will call the clearest pattern extending to the bottom of the image the Primary Pattern, and the one(s) extending to the top of the image the Secondary Pattern(s). Here is the image again, without the box indicating the Primary Pattern (click to zoom):
To choose vertical lines/blobs that define the Primary Pattern, start with the one line/blob that you are most certain is part of the pattern. Always include at least one blob in the Primary Pattern, which will set the level of certainty needed for selecting others. Then add any additional lines/blobs for which you are just about as certain (if any are nearly as certain).
Which is the best list of lines to define the Primary Pattern (bottom)? The "most certain" line/blob is listed first, followed by the others.
To choose vertical lines/blobs that define the Secondary Pattern(s), identify those lines/blobs that are clearly part of pattern besides the Primary Pattern, and that you can assign with the same level of certainty as the ones in the Primary Pattern.
It is possible that all of the vertical lines/blobs are significantly more obscured than the Primary Pattern, in which case it is best not to mark any as definitely belonging to Secondary Pattern(s).
Which is the best list of lines to define the Secondary Pattern(s) (top)?
Final Tips / Rules of Thumb
The following are some rules that you might find helpful for inferring additional information about the patterns you see: