¶ I have a Data Quality question and I don't know where to start? HELP!
The ARM Data Quality Office knows a lot about ARM data. If we don't know the answer to your question we will find the correct person to answer your question. Just give us an e-mail or a call.
ARM Data Quality Office CIWRO, National Weather Center, Suite 2100 University of Oklahoma 120 David L. Boren Blvd. Norman, OK 73072
¶ ARM Data Flow. If I can't see data that should be there, what do I do?
There are a number of ways to determine if data is missing, including looking for plots. Occasionally people will use DQ Plotbrowser to check on data availability. This is a bad idea! The plot browser requires a series of steps to work correctly before the plots will be created, and it is very possible for the data to exist but no plot to be generated. Therefore we recommend using these methods:
Check DQ Dashboard's “Process Status” tab. Entering the datastream name in question in the “Filter by text…” box at the top of the page and the date range to the right will produce a table with daily status of each datastream. The colors of each daily box will correspond to a status (no data file present, data file present but no output, data file present and errors detected, etc). There is a legend in the upper-right hand corner that shows all of the different possible statuses. If there are data files present but no output from that day, you can either email one of the DQO staff or proceed to the method below.
Check DQ Explorer on the day in question. If there is a data file present and no output detected from a days' data processing, a button will pop up in the DQ Explorer metrics table that will submit a processing job to our processing software. Wait several minutes before trying to display the data again. If there is still an issue, send one of the DQO staff an email.
ARM has developed a number of methods to deal with problems. A lot of the methods have come about through a development of a new tool when the need arose. Currently there are a few methods of telling others/documenting a problem.
Check the instrument's Wiki page: Every instrument has a Wiki page, each of which can be found on the homepage or the left sidebar. On each instrument Wiki page, there will be information on expected operations including how plots should look. There is also information on past problems with citations to previous DQPRs. This is a great first check to see if your suspected problem has been documented before.
Email: If you are still unsure if a problem exists, or want clarification on a problem, emailing Ken, Corey, Alyssa, or Randy is a good next step.
DQPR System: Once you are confident that a problem exists, the next step is to issue a DQPR. Note that you can say "this DQPR is being submitted as a preventative check" if you are still unsure if a problem exists! DQPRs allow for the complete tracking of potential problems by including all relevant people (e.g. the instrument mentor, site operations, etc) in the same place. Additionally, it is easy to search for previous DQPRs and document previous problems. If you are unsure if a problem warrants a DQPR, submit one anyway!
After a DQPR is submitted, you will receive an email. You will also receive emails whenever new comments or plots are uploaded to the DQPR. You are encouraged to post updates in the DQPR as appropriate, such as if the problem has gotten better or worse. Generally, it is good to read through all of your DQPRs every week as you go through your DQAs. However, there will be stretches of time when no updates are provided (e.g. such as if spare parts are being sent to an instrument, or if a problem requires quite a bit of time to fix).
DQAs: After issuing a DQPR, don't forget to make note of the problem in your weekly DQA, citing the number of the DQPR you just issued.