Figure 1 shows the conceptual block diagram of Daejeon Correlator. There are several VLBI data playbacks which will be used in our combined VLBI network, such as Mark5B, VERA2000, OCTADISK and optical fiber which shall be connected in near future. Some of them have the VSI-H compatible interface, but the others take the different interface for the data transmission. And they have also their own maximum data recording/playing back rates respectively. To absorb all of these differences and in homogeneity among these existing VLBI data playback systems, the Raw VLBI Data Buffer (RVDB) was introduced, which is a big data server with many large RAID disks and several kind of VLBI data interfaces. The VLBI Correlation Subsystem (VCS) shall receive the VLBI data from the RVDB system, shall calculate the correlation between the every possible pairs of data inputs with proper control parameters given from the monitor and control operation computer, and then shall dump the correlation results into the data archive system. Data archive system, is also a kind of data server, which is used to capture the correlated data output from the VCS and to save them as a structured file system. Finally there is also the correlator control and operation software for overall system.
KVN is now using the Mark5B system for recording and playback the observed data. KASI participated in Mark5B development with Haystack Observatory as a member of international consortium. It can support the VSI compatible and RAID-based HDD storage system. Recording and playing back speed is 1024 Mbps. DIR2000 is widely used in VERA for recording and playback with 1024 Mbps. Recently the manufacturer of DIR2000 had been stopped to manufacture, so NAOJ developed new playing back system named VERA2000, which is modified by DIR1000H system for only supporting playing back. For improving observation and correlation efficiency, VERA uses OCTADISK system for KaVA observations from 2014. So Daejeon Correlator also adopted OCTADISK system, which has the same functions of OCTADDB installed in RVDB (Refer 1.2).
RVDB system, which is developed by NAOJ, consists of Data Input Output (DIO) (currently named OCTAVIA), 10 GbE switch, and Disk Data Buffer (DDB) (currently named OCTADDB). It is able to record the data with maximum 2048 Mbps, and can play back the observed data to the correlator with nominal 2048 Mbps. So, it has a function such as 2048 Mbps recorder and playback at the same time. As shown in Figure 1, the different types of playback systems are used in Daejeon Correlator. So, the purpose of the RVDB system is that it is able to adjust the data format and easily synchronize the data during playback and maintain the buffering between recorder speed and correlation speed.
The main specification of VCS is described in Table 1. The VCS has a capability to process the total 120 cross-correlation and 16 auto-correlation intended for maximum 16 stations, maximum 8192 Mbps (4-streams x 1 Gsps/2-bit/64 MHz clock) input data rates per station. The design architecture for correlator is FX-based and it will use the variable length of FFT (Fast Fourier Transform) to maintain the 0.05 km/sec resolution of velocity at 22 GHz. The maximum delay is ±36,000 km and maximum fringe tracking is 1.075 kHz. The number of frequency channel per correlation output is 8,192.
|# of antennas||16|
|# of inputs/antenna||4 bands(4Fx1P, 2Fx2P, 1Fx2P+2Fx1P)|
|Max. # of corr./input||120 cross + 16 auto|
|Sub-array||2 case(12+4, 8+8)|
|Sampling, Digitization||1 Gsps by 2bit/sample|
|Maximum data rate/antenna||2 Gbps VSI-H (32parallels, 64 MHz clock)|
|Maximum delay compensation||±36,000km|
|Maximum fringe tracking||1.075 kHz|
|Design architecture||FX-type with FPGA|
|FFT word length||16+16 bits fixed point for real, imaginary|
|Integration time||25.6 msec ~ 10.24 sec|
|Data output channels||8192 channels|
|Data output rate||Maximum 1.4GB/sec at 25.6msec integration time|
The basic architecture is a CPU cluster connected with infiniband. For KaVA, the first phase data archive system with about 119TB capacity has been implemented. It has four 10 Gbit Ethernet input port to connect with VCS output and one 10 GbE Ethernet port is connected with data file system for sharing the disk with each other. We have a plan to increase the system capacity for support the EAVN in near future. CODA file system is used in data archive system for making the file system from correlated raw data, which is revised with ccCODA 2.0 library used in Mitaka FX correlator with some modification. New CODA files system based on the ccCODA 2.1 library was installed. Figure 2 shows the view of Daejeon Correlator installed at Korea-Japan Correlation Center (KJCC) located in KASI, Daejeon, Korea.
Figure 2. Daejeon Correlator installed at Korea-Japan Correlation Center.
Daejeon Correlator is currently able to support the following correlation modes.
|Obs. Mode||Total Data Rate||Bandwidth/sub-band||# of sub-bands||Minimum Accum. Time||# of Freq. Channels/sub-band|
|C5||1024 Mbps||16 MHz||16||1.6384 sec||8192|
|C4||1024 Mbps||32 MHz||8||0.8192 sec||8192|
|C3||1024 Mbps||64 MHz||4||0.4096 sec||8192|
|C2||1024 Mbps||128 MHz||2||0.2048 sec||8192|
|C1||1024 Mbps||256 MHz||1||0.1024 sec||8192|
|W1||2048 Mbps||512 MHz||1||0.0512 sec||8192|
|W2||4096 Mbps||512 MHz||2||0.0512 sec||8192|
|W4||8192 Mbps||512 MHz||4||0.0512 sec||8192|
Daejeon Correlator will basically support following frequency channel for preparing FITS file.
KJCC will deliver FITS file to PI by using FTP server or mobile disk.
KJCC will organize the archiving policy for observation data, CODA and FITS file as below.
In order to verify the performance of Daejeon correlator, data analysis was conducted.
For the comparison Daejeon Correlator, Mitaka FX and DiFX correlator were used for data correlation.
Observation experiments were listed in Table 3. We confirmed that results of Daejeon Correlator had
consistent and correct value. Here we will summarize the results for R11027B
Procedure of data analysis was performed as below. And the same parameter was used for all correlation result during analysis. Analysis for spectral-line was adopted with following procedure, but analysis for continuum was excluded for 9 and 10 stages.
|R11027B||Daejeon Correlator evaluation||Continuum, Spectral line||DIR2K, Mark5B||MTK FX, DiFX|
|R11026A||Long time phase monitoring||Continuum||DIR1K, DIR2K, Mark5B||MTK FX|
|K11017A||2 frequency simultaneous observation||Continuum||Mark5B||DiFX|
|K12098C||4 frequency simultaneous observation||Continuum||Mark5B||DiFX|
Figure 3 shows the spectrum shape after completion of FITLD, MSORT, USUBA and INDXR task based on FITS file generated by Daejeon Correlator and DiFX. Source is 3C454.3 for continuum with Yonsei and Ulsan baseline. Phase for all of 16 IF are continuously changed. The phase slope for DiFX is lower than Daejeon Correlator, but we think this is caused by adopting the clock-offset precisely while DiFX correlation. After fringe fitting, these values will be deleted and there is no serious problem in current phase slope.
Figure 3. Spectrum shape of 16IF continuum for Yonsei-Ulsan baseline.
We looked into the variation of GAIN for each source for whole observation time after ACCOR. And then in figure 4, gain amplitude was continuously stable for source with long time integration. But in case of source with short integration time, the gain amplitude pattern for Daejeon Correlator was unstable for time range(between 22:00 and 03:00 for SgrA, Sgr2M) compared with DiFX. This phenomenon was appeared in only KVN station, this is caused with data synchronization while playing back. This phenomenon only occurred beginning of scan about 2~4sec, and the other time range is stable. We recommended the continuous recording of Mark5B while observation.
Figure 4. Variation style of gain amplitude for all stations with 9th IF. Left(Daejeon Correlator), Center(MTK FX), Right(DiFX).
Figure 5 shows the SNR, Delay and Rate for each baseline, respectively. The reference was set as Ulsan station. Integration time for fringe fitting is 30sec, SNR cutoff is 3. We used 3C454.3 which is very strong radio source in continuum sources and phase pattern is clearly remarkable. For comparison with DiFX, all of the patterns for each result are almost similar without any problems.
Figure 5. SNR, Delay, Rate after FRING for 1st IF of Yonsei. Upper(Daejeon Correlator), Bottom(DiFX).
By using previous fringe fitting result, Closure phase for 3C454.3 were obtained and showed in figure 6. We confirmed that closure phase was existed within about 5 degree by calculating with variation of phase for whole observation time and by integrating 50~64 channels value of 1st IF for both Daejeon Correlator and DiFX.
Figure 6. Closure phase after fringe fitting. Upper(Daejeon Correlator), Bottom(DiFX).
Figure 7 shows the spectrum shape after fringe fitting. We draw again the spectrum in order to check phase residual by adopting the value of delay, rate after fringe fitting. For more detail comparison, the range of phase was enlarged to 10 degree and only 8 IFs were shown. Phase residual for spectrum of Daejeon Correlator and DiFX was almost same with 0 value.
Figure 7. Spectrum shape after fringe fitting. Left(Daejeon Correlator), Right(DiFX).
New result file was generated with averaging all IFs, all channels, and whole observation time for tables of 3C454.3 source by using SPLIT task. The 2-dimensional image map was generated using new result file and is shown in figure 8. In figure 8, only KVN 3 stations were used for comparison to DiFX. Flux for Daejeon Correlator is 0.75 Jy/beam, which is less than about 12% compared with 0.84 Jy/beam of DiFX, but the dynamic range is almost same value.
Figure 8. 2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX).
In addition, we plotted the visibility with 1-dimension at UV for confirmation as shown in figure 9. Figure 9 also shows the same result except the difference of flux both 0.75 Jy of Daejeon Correlator and 0.84 Jy of DiFX as plotted in above 2-dimensional map.
Figure 9. Flux density according to UV-distance. Left(Daejeon Correlator), Right(DiFX).
This experiment is to evaluate the performance of 2 frequency simultaneous observation and correlated in November 2012. 8 IF was assigned for 22 and 43 GHz respectively and we will report only 8 IFs for 22 GHz. We looked into the spectrum shape of POSSM after FITLD, MSORT, USUBA and INDXR task. The source is NRAO530, which is strong continuum source and shown in figure 10 for Yonsei-Ulsan baseline. Phase of 8 IFs is continuously changed.
Figure 10. Spectrum shape of 8-IFs continuum for Ulsan-Yonsei baseline. Left(Daejeon Correlator), Right(DiFX).
We looked into the Gain amplitude variation of each source for whole observation time after ACCOR as shown in figure 11. Daejeon Correlator and DiFX had same pattern. This observation was conducted for long time with NRAO150 source, so the loss of 2~4sec at beginning of scan was not affected in correlation result.
Figure 11. Variation style of gain amplitude for each IF of Tamna station. Left(Daejeon Correlator), Right(DiFX).
Figure 12 shows the SNR, Delay and Rate for each baseline, respectively. The reference was set as Ulsan station. Integration time for fringe fitting is 30sec, SNR cutoff is 3. We used 3C454.3 which is very strong radio source in continuum sources and phase pattern is clearly remarkable. For comparison with DiFX, all of the patterns for each result are almost similar without any problems.
Figure 12. SNR, Delay, Rate after FRING for each IF of Tamna station. Left(Daejeon Correlator), Right(DiFX).
By using previous fringe fitting result, Closure phase for NARO150 were obtained and showed in figure 13. We confirmed that closure phase was about 5 degree by calculating with variation of phase for whole observation time and by integrating 20~109 channels value of 1st IF for both Daejeon Correlator and DiFX.
Figure 13. Closure phase after fringe fitting. Upper(Daejeon Correlator), Bottom(DiFX).
We draw again the spectrum in order to check phase residual by adopting the value of delay, rate after fringe fitting as shown in figure 14. For more detail comparison, the range of phase was enlarged to 10 degree and only 8 IFs were shown. Phase residual for spectrum of Daejeon Correlator and DiFX was almost same. And in order to compare the flux, amplitude calibration was also applied. We confirmed that flux of Daejeon Correlator has about 10% lower flux than DiFX as described in R11027B experimental result.
Figure 14. Spectrum shape after fringe fitting and amplitude calibration. Left(Daejeon Correlator), Right(DiFX).
UV coverage or UV plot methods are widely used to confirm how much made up for the UV during observation of NRAO150 source. UV plot is very useful to understand the flux distribution and structure of source. Figure 15 shows the UV coverage and UV plot for Daejeon Correlator result. From pattern of UV plot, we could understand that NRAO150 source is the bright spot with flat in KVN only baseline(less than about 30Mlambda).
Figure 15. The UV coverage (Left) and the amplitude of UV data(Right).
New result file was generated with averaging all IFs, all channels, and whole observation time for tables of 3C454.3 source by using SPLIT task. And then 2-dimensional map was plotted using new result file as shown in figure 16. Flux for Daejeon Correlator is 6.34 Jy/beam, which is less than about 10% compared with 6.83 Jy/beam of DiFX, but the dynamic range is almost same value.
Figure 16. 2-dimensional image map using Difmap. Left(Daejeon Correlator), Right(DiFX).
We confirmed the spectrum shape of POSSM task after completion through FITLD, MSORT, USUBA and INDXR tasks by separating only 9 IF of maser source in R11027B FITS file as shown in figure 17. The used maser source is SgrB2M, which shows strong emission line of H2O of 22 GHz. In case of 9th IF, we conducted the comparison work for 3 kinds of correlator such as Daejeon Correlator, MTK and DiFX. As you can see in figure 17, the entire spectrum shape look like almost same aspect, but Daejeon Correlator has phase concentration in 0 degree at the beginning of bandwidth because of DC-like component. These DC-like component and phase concentration are currently disappeared.
Figure 17. Spectrum shape of 9th IF of Yonsei and Ulsan baseline.
Spectral-line observation such as maser source is occurred the signal at very limited channel(within 10channels) per one maser spot. Therefore we should firstly apply the clock-offset compensation by performing the global fringe fitting according to the bright calibrator source because it is difficult to get the delay in narrow frequency range. In this case, 3C345 source was used for global fringe fitting and figure 18 shows the delay calculated by setting the reference as Tamna station. Daejeon Correlator and MTK had almost same pattern of delay, because same delay parameter was used for correlation. In case of DiFX, it has more small delay value, and we think that this was caused by applying the more detail clock-offset compensation during correlation of DiFX. In figure 18, the delay of correlation result looks good without any problem.
Figure 18. Fringe fitting result of 3C345 as calibrator source.
Fringe fitting was again conducted for maser source by applying the delay value obtained after global fringe fitting. Although fringe fitting for continuum was done in accordance with entire channel, fringe fitting for spectral-line was done with only one channel which has peak value. In this figure, 197th channel has strong flux, so the fringe fitting was done with this channel. Figure 19 shows the variation of SNR of maser source for each station based on Tamna station. Daejeon Correlator, MTK, and DiFX had almost same SNR and variation pattern also look like same.
Figure 19. Fringe fitting result of SgrB2M as maser source, and this figure shows SNR based on Tamna station.
In VLBI observation, the Dopper effect such as the earth rotation or revolution is being affected at each station. Therefore, Dopper effect should be compensated at the data analysis stage. CVEL command of AIPS is used for the Dopper effect compensation. Amplitude calibration was also performed so as to convert power level to real flux value for each station. The results were shown in figure 20. The Dopper effect was successfully compensated for FITS file of 3 correlation result. The unit for each result is Jy unit.
Figure 20. Spectrum shape after fringe fitting, Dopper effect and amplitude calibration.
It is very simple way to confirm the UV coverage in order to know how UV was filled with during observation of SgrB2M maser source. It is helpful to know the structure and distribution of flux of source as UV-distance. Figure 21 shows the UV coverage and UV plot of Daejeon Correlator correlation result. From pattern of UV plot, SgrB2M has the structure of Gaussian distribution at KVN+VERA 7 stations, and it looks like the bright spot with flat in KVN only baseline(less than about 30Mlambda).
Figure 21. UV coverage(Left) and UV plot(Right) of SgrB2M. UV plot looks like Gaussian distribution, but only KVN baseline seems to be flat.
We conducted the imaging work with special channel using maser source as well as continuum source. In case of data processing stage for spectral-line, BPASS and CVEL task were additionally done. The result file was generated by applying result tables for SgrB2M using SPLIT task. For comparison with DiFX, data for only KVN 3 stations was plotted in the same way of continuum source as shown in figure 22. Flux density for Daejeon Correlator is about 10% lower than DiFX, but dynamic range is almost same.
Figure 22. 2-dimensional image map using AIPS. Left(Daejeon Correlator), Right(DiFX).
In case of maser source, maser spots are existed with several tens and thousands in the uniform range. In general, imaging of maser source is performed as follows. Firstly we found each peak channel as refer to spectrum, and then the position of whole maser spots as indicated. Finally 2-dimensional distribution map is plotted. In this experiment, in order to reproduce image, the position was obtained by imaging for several maser spot, and then we confirmed whether the position of maser spot for DiFX result was consistent or not. We conducted the multi-channel imaging work as shown in figure 23 so as to find the maser position, and confirmed that the maser spot for 264th, 269th, 274th, 280th, and 284th channel was detected.
Figure 23. Example of multi-channel image. Image result for each 261th-290th channel.
Daejeon Correlator had solved the recent issue, named the two-layer problem.
This was caused by fault on the address set to data memory reading pointer in data serialize FPGA part.
Resultant data mixtures across the sub-bands was found out as two-layer pattern on a time-power plot,
as like as the left panel of Figure 24.
The influence of two-layer problem is dependent on the power difference to the first sub-band(IF1),
and the loss of visibility amplitude is estimated as less than ~3% for luckyless case.
After fixing that fault, the visibility outputs became stable with no such pattern.
Figure 24. (Left) Before: abnormal shape with two-layer pattern, (Right) After: ordinary (as expected) shape in Gain amplitude plot (KYS, r11027b)