0  4  5  2017-02-01 13:06:03 Report infringing document
OPTIMIZATION OF DISTRIBUTED QUERY USED IN SYNCHRONIZING DATA BETWEEN TABLES WITH DIFFERENT STRUCTURE Demian Horia Faculty of Economics University of Oradea Replication can be used to improve local database performance and to improve the availability of applications. An application can access a local database rather than a central database from another site, which minimize network traffic, locking escalation at the central database and achieve maximum performance for current insert, delete or update operations. The application can continue to function if the central database is down, or cannot be contacted due to a communication problem, power or hardware failure. This paper is focused on presenting a synchronization process between a central Microsoft SQL Server database and many remote sites databases. One possible problem in replication can appear when the two databases have different organization of tables and structures. Keywords: replication, data synchronization, openquery, linked server, parameterized query, uniqueidentifier, SQL Server In this paper we will focus on the problem of synchronizing data between a central database and many client databases, when the client and the server databases have different organization of tables and structures. All the client databases are identical, this means that have the same number of tables, the tables have the same structure, but the records are different. This model can be used to improve local database performance and availability of applications at the clients, because we access the local database rather than a central database. Other benefits are the minimization of network traffic between the client and the central location, the minimization of locking at the central database, the maximization of up time because we does not need a permanent communication channel between the clients and the central location and also the minimization of costs. The architecture of the network The network consists of one Central Server on which we have the central database and many client servers, each of them with its own database used in the local area for storage the local transactions, like in figure 1. Between the client servers and the central server communications channels had been made by using VPN (Virtual Private Network). This communication channels are most of the time down, and from time to time get connected. Client Database Client Server 1 Central Server Central Database Communication channel 810 Client Database Client Server n Figure 1. The architecture of the network If we want to synchronize the information from client database with the server, we have to open first the communication channel and after that to synchronize the data. Microsoft SQL Server offers a mechanism of defining linked servers which can be used to run distributed query against multiple server. Our scenario consists of a one way data transfer from Central Database to the Client Database. We have to obtain new or modified data from the Central Database using inner join operations between TableA and TableB and to copy these data in TableC from Client Database. The fields used for filtering are of uniqueidentifier data type. We want to obtain the best performance for our process of synchronizing the data. One syntax for doing this operation is: Select A.somefields, B.somefields From CentralServer.CentralDatabase.dbo.TableA A Inner Join CentralServer.CentralDatabase.dbo.TableB B On A.idProduct = B.idProduct And B.FilteringField = @SomeValue Where not exists (Select * from TableC C Where C.idProduct = A.idProduct ) In our studying case TableA contains 1915 records, TableB contains 7341 records and TableC contains 1664 records. We obtain 243 new records which have to be copied in TableC. In our test environment this query takes about 47 seconds. The explanation for this bad performance are given to us by the following sentence “Queries involving the following are never delegated to a provider and are always evaluated locally: bit, uniqueidentifier” (Microsoft), which means that the records are taken from the Central Database to the client, and all operations are done at the client. For a poor connection we lose precious time with data transportation. Figure 2. Execution plan The execution plan demonstrates that two remote query are executed against the linked server for bringing locally data from TableA and also from TableB, and only after that the inner join operation are executed at the client site. From this execution plan we understand that 1915 +7341 records are moving from central database to the client site. One thing we have to optimize is that of forcing to execute inner join operation at the Central Server site. This can be done if we define a view in the CentralDatabase or if we use OPENQUERY syntax to send queries which will be executed at the central database. Select * from OPENQUERY(CentralServer,'Select A.somefields, B.somefields from CentralDatabase.dbo.TableA A inner join CentralDatabase.dbo.TableB B on A.idProduct = B.idProduct') As R where R.FilteringField = @SomeValue and not exists (Select * from TableC C Where C.idProduct = A.idProduct ) 811 Because OPENQUERY does not accept variables for its arguments, we have to write filtering condition outside. The execution plan demonstrate us that only 7341 records are coming from the central database which resulted in better performance, which means 43 seconds. Figure 3. Execution plan The next big thing to optimize is that of execution of filtering condition at the central database to obtain only 1907 records instead of 7341. If we know exactly the value used for filtering we can write in the OPENQUERY the filtering condition. Select * from OPENQUERY(CentralServer,'Select A.somefields, B.somefields from CentralDatabase.dbo.TableA A inner join CentralDatabase.dbo.TableB B on A.idProduct = B.idProduct and A.FilteringField =VALUE ') As R where not exists (Select * from TableC C Where C.idProduct = A.idProduct ) The time needed for executing this query was 7 seconds, and the execution plan can be analyze from the figure 4. Figure 4. Execution plan As we can see only 1907 records came from the central database, which is an explanation for our best results until now. Because OPENQUERY does not accept variables for its argument, we can construct the sting dynamically and executes after that like in the following example: DECLARE @SQLSTMT nCHAR(4000) DECLARE @SOMEVALUE uniqueidentifier --we can put here a code for initializing the value of our variables SET @SOMEVALUE = '341FE6DA-D431-4D72-93F0-E20B2CDB1767' SET @SQLSTMT =N' Select * ' +'from OPENQUERY ( CENTRALSERVER, ''Select A.somefields, B.somefields ' +' from CentralDatabase.dbo.TableA A' +' inner join CentralDatabase.dbo.TableB B ' 812 +' on A.idProduct = B.idProduct ‘ +’ and B.FilteringField = '''''+cast(@idSomeValue as char(36))+''''''') As A ' +' where not exists ( select * from TableC C ' +' where C.idProduct = A.idProduct) ' EXECUTE SP_EXECUTESQL @SQLSTMT The execution plan for this command can be viewed in the following figure, and we can see the same results as in the last example. The execution of the Join operation and the filtering occurs on the central server. We achieve the same results. Figure 5. Execution plan for dynamic query The speed is well for the preceding examples, but we can improve performance, if we can rewrite the query without making reference to a table from the client site. To do this we have to kow for every records from the central site for every records involved in the query, the date when the record was added or modified. This date has to be copied to the client site. DECLARE @SQLSTMT DECLARE @SOMEVALUE DECLARE @DATA nCHAR(4000) uniqueidentifier datetime SET @SOMEVALUE = '391FE6DA-D431-4D72-93F0-E20B2CDB1767' SELECT @DATA =MAX(DATA) FROM TABLEC SET @SQLSTMT =N' Select * ' +'from OPENQUERY ( CENTRALSERVER, ''Select A.somefields, B.somefields ' +' from CentralDatabase.dbo.TableA A' +' inner join CentralDatabase.dbo.TableB B ' +' on A.idProduct = B.idProduct ‘ +’ and B.FilteringField = '''''+cast(@SomeValue as char(36))+'''''' +' and P.Dataoperarii >='''''+cast(@data as char(20))+'''''''' +') As A ' EXECUTE SP_EXECUTESQL @SQLSTMT We obtain 1 seconds for 243 records, the best results. Figure 6.Execution plan on the linked server 813 Another benefit of dynamically creation of sql commands consist of construction of a mechanism for calling function from linked server, like we can see in the following example: DECLARE @SQLSTMT nCHAR(4000) DECLARE @idParameter uniqueidentifier --we can put here a code for initializing the value of our variables SET @ idParameter = '6EDAE050-F5EA-4B1C-823B-28B8F599FDC9' SET @SQLSTMT =N' Select * ' +'from OPENQUERY(DOUAROTI,''select * ' +' from DataBase.dbo.FunctionName( '''''+cast(@idParameter as char(36))+''''')'' ) ' --we used only apostrophes EXECUTE SP_EXECUTESQL @SQLSTMT Conclusion: 1. Even if OPENQUERY does not accept variables for its argument we can build dynamically the command and executed against the linked server. 2. For better performance we have to limit the number of records which are transported from one server to other. 3. It’s better to force the execution of JOIN operation at the sites where the tables are located, if we need better performance 4. Using OPENQUERY we can call function from the linked server, which can used variables. 5. The best results can be achieved if all operations will be executed on the central server. Bibliography: 26. Microsoft. (n.d.). Optimizing Distributed Queries. Retrieved may 20, 2010, from msdn: http://msdn.microsoft.com/en-us/library/Aa178113 27. Support, M. (2003, October 3). PRB: User-Defined Function Call in Four-Part Linked Server Query Fails with . Retrieved April 20, 2010, from Support: http://support.microsoft.com/kb/319138 28. Team, Q. O. (2006, April 6). Tips, Tricks, and Advice from the SQL Server Query Optimization Team. Retrieved May 20, 2010, from msdn: http://blogs.msdn.com/queryoptteam/ 814 equilibrium value of MeCpG steps (,+14 deg.) [31,44]. In comparison, methylation has a significantly lower stability cost when happening at major groove positions, such as 211 and 21 base pair from dyad (mutations 9 and 12), where the roll of the nucleosome bound conformation (+10 deg.) is more compatible with the equilibrium geometry of MeCpG steps. The nucleosome destabilizing effect of cytosine methylation increases with the number of methylated cytosines, following the same position dependence as the single methylations. The multiple-methylation case reveals that each major groove meth- PLOS Computational Biology | www.ploscompbiol.org 3 November 2013 | Volume 9 | Issue 11 | e1003354 DNA Methylation and Nucleosome Positioning ylation destabilizes the nucleosome by around 1 kJ/mol (close to the average estimate of 2 kJ/mol obtained for from individual methylation studies), while each minor groove methylation destabilizes it by up to 5 kJ/mol (average free energy as single mutation is around 6 kJ/mol). This energetic position-dependence is the reverse of what was observed in a recent FRET/SAXS study [30]. The differences can be attributed to the use of different ionic conditions and different sequences: a modified Widom-601 sequence of 157 bp, which already contains multiple CpG steps in mixed orientations, and which could assume different positioning due to the introduction of new CpG steps and by effect of the methylation. The analysis of our trajectories reveals a larger root mean square deviation (RMSD) and fluctuation (RMSF; see Figures S2– S3 in Text S1) for the methylated nucleosomes, but failed to detect any systematic change in DNA geometry or in intermolecular DNA-histone energy related to methylation (Fig. S1B, S1C, S4–S6 in Text S1). The hydrophobic effect should favor orientation of the methyl group out from the solvent but this effect alone is not likely to justify the positional dependent stability changes in Figure 2, as the differential solvation of the methyl groups in the bound and unbound states is only in the order of a fraction of a water molecule (Figure S5 in Text S1). We find however, a reasonable correlation between methylation-induced changes in hydrogen bond and stacking interactions of the bases and the change in nucleosome stability (see Figure S6 in Text S1). This finding suggests that methylation-induced nucleosome destabilization is related to the poorer ability of methylated DNA to fit into the required conformation for DNA in a nucleosome. Changes in the elastic deformation energy between methylated and un-methylated DNA correlate with nucleosomal differential binding free energies To further analyze the idea that methylation-induced nucleosome destabilization is connected to a worse fit of methylated DNA into the required nucleosome-bound conformation, we computed the elastic energy of the nucleosomal DNA using a harmonic deformation method [36,37,44]. This method provides a rough estimate of the energy required to deform a DNA fiber to adopt the super helical conformation in the nucleosome (full details in Suppl. Information Text S1). As shown in Figure 2, there is an evident correlation between the increase that methylation produces in the elastic deformation energy (DDE def.) and the free energy variation (DDG bind.) computed from MD/TI calculations. Clearly, methylation increases the stiffness of the CpG step [31], raising the energy cost required to wrap DNA around the histone octamers. This extra energy cost will be smaller in regions of high positive roll (naked DNA MeCpG steps have a higher roll than CpG steps [31]) than in regions of high negative roll. Thus, simple elastic considerations explain why methylation is better tolerated when the DNA faces the histones through the major groove (where positive roll is required) that when it faces histones through the minor groove (where negative roll is required). Nucleosome methylation can give rise to nucleosome repositioning We have established that methylation affects the wrapping of DNA in nucleosomes, but how does this translate into chromatin structure? As noted above, accumulation of minor groove methylations strongly destabilizes the nucleosome, and could trigger nucleosome unfolding, or notable changes in positioning or phasing of DNA around the histone core. While accumulation of methylations might be well tolerated if placed in favorable positions, accumulation in unfavorable positions would destabilize the nucleosome, which might trigger changes in chromatin structure. Chromatin could in fact react in two different ways in response to significant levels of methylation in unfavorable positions: i) the DNA could either detach from the histone core, leading to nucleosome eviction or nucleosome repositioning, or ii) the DNA could rotate around the histone core, changing its phase to place MeCpG steps in favorable positions. Both effects are anticipated to alter DNA accessibility and impact gene expression regulation. The sub-microsecond time scale of our MD trajectories of methylated DNAs bound to nucleosomes is not large enough to capture these effects, but clear trends are visible in cases of multiple mutations occurring in unfavorable positions, where unmethylated and methylated DNA sequences are out of phase by around 28 degrees (Figure S7 in Text S1). Due to this repositioning, large or small, DNA could move and the nucleosome structure could assume a more compact and distorted conformation, as detected by Lee and Lee [29], or a slightly open conformation as found in Jimenez-Useche et al. [30]. Using the harmonic deformation method, we additionally predicted the change in stability induced by cytosine methylation for millions of different nucleosomal DNA sequences. Consistently with our calculations, we used two extreme scenarios to prepare our DNA sequences (see Fig. 3): i) all positions where the minor grooves contact the histone core are occupied by CpG steps, and ii) all positions where the major grooves contact the histone core are occupied by CpG steps. We then computed the elastic energy required to wrap the DNA around the histone proteins in unmethylated and methylated states, and, as expected, observed that methylation disfavors DNA wrapping (Figure 3A). We have rescaled the elastic energy differences with a factor of 0.23 to match the DDG prediction in figure 2B. In agreement with the rest of our results, our analysis confirms that the effect of methylation is position-dependent. In fact, the overall difference between the two extreme methylation scenarios (all-in-minor vs all-in-major) is larger than 60 kJ/mol, the average difference being around 15 kJ/ mol. We have also computed the elastic energy differences for a million sequences with CpG/MeCpG steps positioned at all possible intermediate locations with respect to the position (figure 3B). The large differences between the extreme cases can induce rotations of DNA around the histone core, shifting its phase to allow the placement of the methylated CpG steps facing the histones through the major groove. It is illustrative to compare the magnitude of CpG methylation penalty with sequence dependent differences. Since there are roughly 1.5e88 possible 147 base pairs long sequence combinations (i.e., (4n+4(n/2))/2, n = 147), it is unfeasible to calculate all the possible sequence effects. However, using our elastic model we can provide a range of values based on a reasonably large number of samples. If we consider all possible nucleosomal sequences in the yeast genome (around 12 Mbp), the energy difference between the best and the worst sequence that could form a nucleosome is 0.7 kj/mol per base (a minimum of 1 kJ/mol and maximum of around 1.7 kJ/mol per base, the first best and the last worst sequences are displayed in Table S3 in Text S1). We repeated the same calculation for one million random sequences and we obtained equivalent results. Placing one CpG step every helical turn gives an average energetic difference between minor groove and major groove methylation of 15 kJ/ mol, which translates into ,0.5 kJ/mol per methyl group, 2 kJ/ mol per base for the largest effects. Considering that not all nucleosome base pair steps are likely to be CpG steps, we can conclude that the balance between the destabilization due to CpG methylation and sequence repositioning will depend on the PLOS Computational Biology | www.ploscompbiol.org 4 November 2013 | Volume 9 | Issue 11 | e1003354 DNA Methylation and Nucleosome Positioning Figure 3. Methylated and non-methylated DNA elastic deformation energies. (A) Distribution of deformation energies for 147 bplong random DNA sequences with CpG steps positioned every 10 base steps (one helical turn) in minor (red and dark red) and major (light and dark blue) grooves respectively. The energy values were rescaled by the slope of a best-fit straight line of figure 2, which is 0.23, to por la lectura a través de la lectura de la prensa. La educación en los medios las fuerzas dispersas en función de los soportes mediáticos y orientarse más hacia la educación en medios que al dominio adquiere pleno derecho y entidad en la sección sexta titulada «competencias sociales y cívi- técnico de los aparatos. cas» que indica que «los alum- nos deberán ser capaces de juz- gar y tendrán espíritu crítico, lo que supone ser educados en los las programaciones oficiales, ya que, a lo largo de un medios y tener conciencia de su lugar y de su influencia estudio de los textos, los documentalistas del CLEMI en la sociedad». han podido señalar más de una centena de referencias a la educación de los medios en el seno de disciplinas 4. Un entorno positivo como el francés, la historia, la geografía, las lenguas, Si nos atenemos a las cifras, el panorama de la las artes plásticas : trabajos sobre las portadas de educación en medios es muy positivo. Una gran ope- prensa, reflexiones sobre temas mediáticos, análisis de ración de visibilidad como la «Semana de la prensa y publicidad, análisis de imágenes desde todos los ángu- de los medios en la escuela», coordinada por el CLE- los, reflexión sobre las noticias en los países europeos, MI, confirma año tras año, después de 17 convocato- información y opinión rias, el atractivo que ejerce sobre los profesores y los Esta presencia se constata desde la escuela mater- alumnos. Concebida como una gran operación de nal (2 a 6 años) donde, por ejemplo, se le pregunta a complementariedad entre la escuela y los profesiona- los niños más pequeños si saben diferenciar entre un les de los medios, alrededor del aprendizaje ciudada- periódico, un libro, un catálogo, a través de activida- no de la comunicación mediática, este evento moviliza des sensoriales, si saben para qué sirve un cartel, un durante toda una semana un porcentaje elevado de periódico, un cuaderno, un ordenador si son capa- centros escolares que representan un potencial de 4,3 ces de reconocer y distinguir imágenes de origen y de millones de alumnos (cifras de 2006). Basada en el naturaleza distintas. Podríamos continuar con más voluntariado, la semana permite desarrollar activida- ejemplos en todos los niveles de enseñanza y práctica- des más o menos ambiciosas centradas en la introduc- Páginas 43-48 ción de los medios en la vida de la escuela a través de la instalación de kioscos, organización de debates con profesionales y la confección por parte de los alumnos de documentos difundidos en los medios profesionales. Es la ocasión de dar un empujón a la educación en medios y de disfrutarlos. Los medios –un millar en 2006– se asocian de maneras diversas ofreciendo ejemplares de periódicos, acceso a noticias o a imágenes, proponiendo encuentros, permitiendo intervenir a los jóvenes en sus ondas o en sus columnas Esta operación da luz al trabajo de la educación en medios y moviliza a los diferentes participantes en el proyecto. 5. La formación de los docentes La formación es uno de los pilares principales de la educación en los medios. Su función es indispensable ya que no se trata de una disciplina, sino de una enseñanza que se hace sobre la base del voluntariado y del compromiso personal. Se trata de convencer, de mostrar, de interactuar. En primer lugar es necesario incluirla en la formación continua de los docentes, cuyo volumen se ha incrementado desde 1981 con la aparición de una verdadera política de formación continua de personal. Es difícil dar una imagen completa del volumen y del público, pero si nos atenemos a las cifras del CLEMI, hay más de 24.000 profesores que han asistido y se han involucrado durante 2004-05. 5.1. La formación continua En la mayoría de los casos, los profesores reciben su formación en contextos cercanos a su centro de trabajo, o incluso en este mismo. Después de una política centrada en la oferta que hacían los formadores, se valora más positivamente la demanda por parte del profesorado, ya que sólo así será verdaderamente fructífera. Los cursos de formación se repartieron en varias categorías: desde los formatos más tradicionales (cursos, debates, animaciones), hasta actividades de asesoramiento y de acompañamiento, y por supuesto los coloquios que permiten un trabajo en profundidad ya que van acompañados de expertos investigadores y profesionales. Citemos, por ejemplo en 2005, los coloquios del CLEMI-Toulouse sobre el cine documental o el del CLEMI-Dijon sobre «Políticos y medios: ¿connivencia?». Estos coloquios, que forman parte de un trabajo pedagógico regular, reagrupan a los diferentes participantes regionales y nacionales alrededor de grandes temas de la educación en medios y permiten generar nuevos conocimientos de aproximación y una profundización. Páginas 43-48 Hay otro tipo de formación original que se viene desarrollando desde hace menos tiempo, a través de cursos profesionales, como por ejemplo, en el Festival Internacional de Foto-periodismo «Visa para la imagen», en Perpignan. La formación se consolida en el curso, da acceso a las exposiciones, a las conferencias de profesionales y a los grandes debates, pero añade además propuestas pedagógicas y reflexiones didácticas destinadas a los docentes. Estas nuevas modalidades de formación son también consecuencia del agotamiento de la formación tradicional en las regiones. Los contenidos más frecuentes en formación continua conciernen tanto a los temas más clásicos como a los cambios que se están llevando a cabo en las prácticas mediáticas. Así encontramos distintas tendencias para 2004-05: La imagen desde el ángulo de la producción de imágenes animadas, el análisis de la imagen de la información o las imágenes del J.T. La prensa escrita y el periódico escolar. Internet y la información en línea. Medios y educación de los medios. 5.2 La formación inicial La formación inicial está aun en un grado muy ini- cial. El hecho de que la educación en medios no sea una disciplina impide su presencia en los IUFM (Institutos Universitarios de Formación de Maestros) que dan una prioridad absoluta a la didáctica de las disciplinas. En 2003, alrededor de 1.400 cursillistas sobre un total de 30.000 participaron en un momento u otro de un módulo de educación en medios. Estos módulos se ofrecen en función del interés que ese formador encuentra puntualmente y forman parte a menudo de varias disciplinas: documentación, letras, historia-geografía Estamos aún lejos de una política concertada en este dominio. La optativa «Cine-audiovisual» ha entrado desde hace muy poco tiempo en algunos IUFM destinada a obtener un certificado de enseñanza de la opción audiovisual y cine. Internet tiene cabida también en los cursos de formación inicial, recientemente con la aparición de un certificado informático y de Internet para los docentes, dirigido más a constatar competencias personales que a valorar una aptitud para enseñarlos. 6. ¿Y el futuro? El problema del futuro se plantea una vez más por la irrupción de nuevas técnicas y nuevos soportes. La difusión acelerada de lo digital replantea hoy muchas cuestiones relativas a prácticas mediáticas. Muchos Comunicar, 28, 2007 47 Comunicar, 28, 2007 Enrique Martínez-Salanova '2007 para Comunicar 48 trabajos que llevan el rótulo de la educación en medios solicitan una revisión ya que los conceptos cambian. La metodología elaborada en el marco de la educación en medios parece incluso permitir la inclinación de la sociedad de la información hacia una sociedad del conocimiento, como defiende la UNESCO. En Francia, se necesitaría unir las fuerzas dispersas en función de los soportes mediáticos y orientarse más hacia la educación en medios que al dominio técnico de los aparatos. Los avances recientes en el reconocimiento de estos contenidos y las competencias que supondrían podrían permitirlo. Referencias CLEMI/ACADEMIE DE BORDEAUX (Ed.) (2003): Parcours médias au collège: approches disciplinaires et transdisciplinaires. Aquitaine, Sceren-CRDP. GONNET, J. (2001): Education aux médias. Les controverses fécondes. Paris, Hachette Education/CNDP. SAVINO, J.; MARMIESSE, C. et BENSA, F. (2005): L’éducation aux médias de la maternelle au lycée. Direction de l’Enseignement Scolaire. Paris, Ministère de l’Education Nationale, Sceren/CNDP, Témoigner. BEVORT, E. et FREMONT, P. (2001): Médias, violence et education. Paris, CNDP, Actes et rapports pour l’éducation. – www.clemi.org: fiches pédagogiques, rapports et liens avec les pages régionales/académiques. – www.ac-nancy-metz.fr/cinemav/quai.html: Le site «Quai des images» est dédié à l’enseignement du cinéma et de l’audiovisuel. – www.france5.fr/education: la rubrique «Côté profs» a une entrée «education aux médias». – www.educaunet.org: Programme européen d’éducation aux risques liés à Internet. dResedfeleexliobnuetsacón Páginas 43-48
123dok avatar

Ingressou : 2016-12-29

Documento similar
Replication Data Synchronization


Linked Server

Parameterized Query


Sql Server