Regular Article – Experimental Physics THE EUROPEAN PHYSICAL JOURNAL C Search for lepton flavour violation in ep collisions at HERA The H1 Collaboration

A search for the lepton flavour violating processes ep → µX and ep → τ X is performed with the H1 experiment at HERA. Final states with a muon or tau and a hadronic jet are searched for in a data sample corresponding to an integrated luminosity of 66.5 pb −1 for e + p collisions and 13.7 pb −1 for e − p collisions at a centre-of-mass energy of 319 GeV. No evidence for lepton flavour violation is found. Limits are derived on the mass and the couplings of leptoquarks inducing lepton flavour violation in an extension of the Buchmüller–Rückl–Wyler effective model. Leptoquarks produced in ep collisions with a coupling strength of λ = 0.3 and decaying with the same coupling strength to a muon–quark pair or a tau–quark pair are excluded at 95% confidence level up to masses of 459 GeV and 379 GeV, respectively.


where Q 2 = s
y refers to the generated negative momentum transfer squared and x is the Bjørken scaling variable known at the generator level.This procedure provides an exact prediction over the full range of LQ production parameters and avoids approaches like the narrow width approximation or the high mass (contact interaction) approximation.

The LQ kinematics are reconstructed using the double angle method [16].The direction of the detected lepton and jet are used to reconstruct t e Bjørken scaling variable x and therefore the LQ mass m rec LQ = √ xs.

The contributions from Standard Model (SM) background processes which may mimic the signal include neutral current (NC) and charged current (CC) deep-inelastic scattering (DIS), photoproduction, lepton pair production and real W boson production.These processes are briefly described below:
• NC DIS (ep → eX)
NC DIS processes contribute to the selected event sa ple if the scattered electron is attributed to the tau electronic decay or if it is misidentified as a narrow jet corresponding to a tau decay to hadrons.The NC DIS background is modelled using the event generator RAPGAP [17].The proton PDFs are parametrised using CTEQ5L [15] and hadronisation is performed using JETSET [18] parton showers and the Lund string fragmentation.

• CC DIS (ep → νX) Lepton flavour violatin

processes usually exhibi
an imbalance in the measured calorimetric transverse momentum due to either the presence of a minimally ionising muon in µX final states or the escaping neutrino(s) from tau decays in τ X events.This imbalance is exploited in the event selection.The CC DIS process leads to events with genuine missing transverse momentum and therefore contributes to the selected sample if hadrons or photons from the final state are misidentified as muons or if tau decays are fals ly reconstructed.The CC DIS contribution is modelled using the DJANGO event generator [19].

• Photoproduction (γp → X) Events from photoproduction processes may contribute to the final selection if a hadron is wrongly identified as a muon or if a narrow hadronic jet fakes the tau signature.This contribution is calculated using the event generator PYTHIA [20].CTEQ5L [15] serves as the proton PDF parametrisation and the photonic parton distribution parametrisation GRV-LO [21] is used.As PYTHIA only contains leading order 2 → 2 processes, the multi-jet production cross section is underestimated [22].Therefore, the prediction is scaled up by a factor 1.2 in this analysis, in agreement with previous analyses of j ts in photoproduction [22].

• Lepton-pair production (ep → eℓ + ℓ − X) Lepton-pair production events contribute to the background because they may lead to high momentum leptons in the final state.In particular, inelastic di-muon events with one unidentified muon may fake the µX LFV signature.The background samples include ee, µµ and τ τ production generated with the event generator GRAPE [23].

• W production (ep → eW X) Real W boson production leads to final states with isolated high P T leptons and missing transverse momentum.The simulated W production samples are created w th the event generator EPVEC [24] and include leptonic (eν e , µν µ , τ ντ ) and hadronic W decays.

All signal and SM samples are passed through a detailed simulation of the H1 detector response based on the GEANT program [25] and the same reconstruction and analysis algorithms as used for the data.


High P T Muon Signatures

Leptoquarks with couplings to the first and the second lepton genera ion can be produced in ep collisions and may decay to a muon and a quark.The signature is an isolated high P T muon back-to-back to the hadronic system in the tr nsverse plane.In general, a muon deposits a very small fraction of its energy in the LAr calorimeter.The signal is therefore expected to exhibit large P calo T , which is the net transverse momentum reconstructed from all clusters recorded in the LAr calorimeter alone.

The event preselection requires at least one muon with a transverse momentum above 10 GeV in the polar angular range 10 • to 140 • and at least one jet.The muon is required to be isolated.The angular distance, D = (∆η) 2 + (∆φ) 2 , of the muon to the nearest track and to the nearest jet is required to be greater than 0.5 and 1.0, respectively.Only events with P calo T greater than 12 GeV are selected.In order to furthe

exploit the event topology in the transverse p
ane, the cut V ap /V p < 0.3 is employed, where V ap /V p is defined as the ratio of the anti-parallel to parallel projec ions of all energy deposits in the calorimeter with respect to the direction of P calo T [26].

Figure 2 displays the distributions of the transverse momentum of the muon, its polar angle θ µ , P calo T and the acoplanarity ∆φ µ−X between the muon and the hadronic final state X after the muon preselection.The data passing the preselection are well described by the SM prediction.The signal corresponding to a scalar LQ with m LQ = 200 GeV is also shown.It displays muons with large P µ T produced predominantly in the forward direction (low θ µ ) in events with significant P calo T and back-to-back topology ∆φ µ−X ≃ 180 • .

In the final LFV selection step, the NC DIS background is further suppressed by rejecting events with identified electrons, and by accepting only events with an imbalance of the calorimeter deposits, P calo T > 25 GeV, and with a back-to-back topology, ∆φ µ−X > 170 • .The latter selection criterion is only applied for events for which the hadronic final state is well contained in the detector, with the reconstructed polar angle 7
• < θ X < 140 • .
The selection efficiency ranges from 40% to 60% depending on the LQ mass and type (see table 2).


High P T Tau Signatures

Leptoquarks with couplings to the first and the third lepton generation can be produced in ep collisions and may decay to a tau and a quark.Tau leptons are identified using the electronic, muonic and hadronic decays of the tau.


Electronic tau decays

The final state resulting from the electronic tau decay, τ → eν e ν τ , leads to an event topology that is very similar to that of high Q 2 NC DIS events.The preselection follows that presented in [27].A reconstructed jet with a minimal transverse momentum of P j T > 25 GeV backto-back in the transverse plane to an electron with P e T > 10 GeV is required.The kinematic domain is restricted to Q 2 > 1000 GeV 2 and y > 0.1.Figure 3(a) shows the distribution of P miss T after this preselection, where P miss T is defined as the total In the final selection a large missing transverse momentum P miss T > 20 GeV is re nce between the transverse momenta of the electron P e T and the hadronic final state P X T .Hence, the restriction P e T /P X T < 0.8 further reduces NC DIS background.In addition, the azimuthal distance between the missing transverse momentum and th electron must not exceed 20 • .The remaining NC DIS background, due to mismeasured electron energies leading to missing energy near the electron, is reduced by the requirement P e−clu T /P e−trk T > 0.7, where P e−clu T is measured from the electromagnetic cluster and P e−trk T from tracking informat nal selection in the electronic tau decay channel yields an efficiency normalised to all tau decays of 3% to 10%, which is limited by the branching fraction BR(τ → eν e ν τ ) = 17.8% [28] and dependent on the assumed LQ mass and type (see table 2).


Muonic tau decays

Muonic tau decays τ → µν µ ν τ result in similar final states as the high P T muon signatures described in section 4. The same selection cuts described therein are applied here.To account for possible effects due to different uon kinematics resulting from a tau decay, the selection efficiency was studied in detail with a LFV MC signal sample with a τ X final state and a subsequent muonic tau decay.The selection efficiency ranges between 4% and 8%, which is dependent on the LQ mass and type, normalised to all tau decays and limited by BR(τ → µν µ ν τ ) = 17.4% [28] (see table 2).


Hadronic tau decays

The hadronic decays of the high P T tau lead to a typical signature of a high P T "pencil-like" jet.The signal topology is a di-jet event with no leptons.The tau-jet is characterised by a narrow energy deposit in the calorimeter and low track multiplicity with predominantly one or three tracks in the identification cone of the jet.The neutrino from the tau decay are boosted along the direction of the hadrons.The missing transverse momentum in the event is ng angle that varies between 5 • and 30 • with decreasing jet momentum.The tracks are required not to be associated with identified electrons or muons and the scalar sum of their transverse momenta is required to be larger than 2 GeV.The fine granularity of the LAr calorimeter is used to match extrapolated tracks with energy deposits in the calo imeter and to separate additional neutral particles associated to the tau candidate from unmatched energy deposits in the tau-jet cone.The sum of the four-vectors of the tracks and of the neutral particles defines t e tau-jet candidate four-vector.

In the preselection step at least two jets with a transverse momentum P jet1 T > 20 GeV and P jet2 T > 15 GeV reconstructed in the polar angle range 7 • < θ jets < 145 • are required.One jet must fulfil the criteria of a tau-jet candidate with θ τ jet > 20 • calorimetric shower shape and tracking signature are exploited to validate the tau-jet candidates.The following estimators are used to separate a tau-jet from quark or gluon induced jets: the number of all tracks associated to the tau-jet candidate, the distance in η − φ between tracks and calorimetric clusters, the number of calorimeter cells of the tau-jet n cells , the radial extension of the calorimetric deposits r = n cell i=1 E i r i / i E i , the standard deviation σ(r) = r 2 − r 2 and the invariant tau-jet mass reconstructed from calorimeter cells.A neural net algorithm is employed and trained using the six estimator variables, as explained in [29].The neural net yields a discriminator variable D N N in the range 0 ≤ D NN ≤ 1 with values close to 0 for quark or gluon induced jets and close to 1 for hadronic tau decays.The distribution of the discriminant D N N after the preselection is depicted in figure 3(b).The distributions of P miss T and ∆φ miss−τ jet after requiring D N N > 0.8 are shown in figures 3(c),(d).This requirement yields a ignal efficiency of 80% and a quark or gluon induced jet rejection of 95%.After all preselection criteria 16 (112) events are selected in e − p (e + p) data sample for 22.0 ± 1.0(stat.)(121.1 ± 5.3(stat.))expected from the SM.

The final selection step in the hadronic tau decay channel makes use of the characteristic large missing transverse momentum carried by the tau neutrino which is expected to be in the direction of the tau-jet.The difference in φ between the missing transverse momentum vector and the tau-jet, ∆φ miss−τ jet , is required to be below 20 • .A minimal value of P miss T > 12 GeV is chosen for an accurate determination of the directi n.In addition P calo T > 12 GeV is required.The final signal selection efficiency in the hadronic tau decay channel varies between 3% and 13%, normalised to all tau decays and limited by the branching fraction BR(τ − → ν τ + hadrons) = 64.8%[28] (see table 2).


Systematic Uncertainties

The following experimental systematic uncertainties are considered:

• The energy f electrons is measured with a systematic uncertainty in the range from 0.7% to 3% depending on the polar angle.The uncertainty of the electron direction is estimated to be less than 3 mrad in θ and 1 mrad in φ.

• The scale uncertainty on the transverse momentum of high P T muons amou ts to 5%.

The uncertainty on the reconstruction of the muon direction is 3 mrad in θ and 1 mrad in φ.

• For the hadronic final state, an energy scale uncertainty of 2% and a direction uncertainty of

mrad are assumed.

• The
luminosity of the analysed datasets is known to 1.5%.

The effects of these systematic uncertainties on the signal and the expected SM background are evaluated by shifting the relevant quantities in the MC simulation by their uncertainty and adding all resulting variations in quadrature.

Systematic errors accounting for normalisation uncertainties on the expected background determined from the individual MC event generators are estimated to be 10% for NC DIS and Lepton-pair production, 15% for W pr duction and 30% for photoproduction and CC DIS.The relatively large error of 30% on photoproduction and CC DIS is due to uncertainties on higher-order corrections.The errors associated to the background normalisation are added in quadrature to the experimental error to calculate the total error of the SM prediction.

The main theoretical uncertainty on the signal cross section originates from the parton densities.This uncertainty is estimated as described in [13].It is found to be 5% for LQs coupling to up-type quarks and varies between 7% at low masses and 30% at masses around 290 GeV for LQs coupling to down-type quarks.The correlation between different channels is taken into account for the statistical interpretation and limit calculation [30].A detailed description of the analysis can be found in [31].


Results

No candidate is found in the final data sample of the muon channel.The expected number of SM background events is 1.03 ± 0.32 in the e + p set and 0.18 ± 0.06 in the e − p sample.The largest contribution to this background comes from muon-pair production and the muonic decays of W bosons.These results apply equally to the muonic tau decay channel.

In the electronic tau decay channel no data eve t is found compared to a SM expectation of 0.28 ± 0.19 events in the e − p sample and 1.24 ± 0.55 events in the e + p data.NC DIS events with a mismeasured electron energy are the largest background contribution.

No e − p data event passes the final selection criteria in the hadronic tau decay channel.The expected SM background amounts to 0.29 ± 0.06.One event is selected in the e + p data for an expected SM prediction of 2.63 ± 0.57, dominated by C DIS and photoproduction processes.

The results of the final selection in all channels are summ

ised in table 2. Typical
signal selection efficiencies for some LQ types with a mass of 150 GeV and 500 GeV are also given.The observation is in agreement with the SM prediction and no evidence for LFV is found by the present analysis.Limits on the model

arameters presented in
section 3 are calculated as described in the following section.


Limits

The results of the search are interpreted in terms of exclusion limits on the mass and the coupling of LQs that may mediate LFV.The LQ production mechanism at HERA involves non-zero coupling to the first generation fermions λ eq > 0. The LFC decays LQ → eq or LQ → ν e q are therefore possible.In order to cover the full LQ decay width and to generalise the results of LFV searches in ep collisions to an arbitrary weight between the LFC and LFV decay channels, the searches for LFC decays presented in [13] are combined with each of the LFV search channels µX or τ X of the present analysis.It is assumed that only one of the couplings λ µq and λ τ q is non-zero and therefore the µX and τ X channels enter the limits calculation separately.A modified frequentist method with a likelihood ratio as the test statistic is used to combine the individual data sets and search channels [32].

In first generation LQ signals are searched for in about 400 bins in the m LQ −y plane and the observed data is in agreement with the irreducible SM NC and CC background [13].For the LFV channel µq (τ q), the couplings λ eq and λ µq (λ τ q ) and the LQ mass determine the total production cross section, which is compared to the selected data from the LFV search channel and the first generation results.A combined test statistic is built and used to set limits as a function of λ eq , λ µq (λ τ q ) and m LQ .This procedure implicitly includes in the analysis the decays to a neutrino of any flavour and a quark.

Figure 4 shows limits before and after combination with the search for first generation LQs for the LQ types S L 0 and V L 0 up to LQ masses of 320 GeV assuming λ eq = λ µq and λ eq = λ τ q , i.e. β LFV = 0.5, in the

esonance productio
region.The comparison for these types exemplifies that the limits on those LQs which can decay to a neutrino-quark pair, namely S L 0 , S L 1 , V L 0 and V L 1 , benefit most from the combination with the search for first generation LQs which covers decays to a neutrino-quark pair.In the high mass regime m LQ ≫ √ s (contact interaction region) the obtained limits are similar to those deduced without the combination.The fluctuations in the combined limits are due to the observed data events in the search for first generation LQs.In the mass range from 250 GeV to 300 GeV both the com

ned limits on λ µq a
d λ τ q are for all LQ types up to a factor 2 more stringent than without combination.Table 3 shows the 95% CL combined lower limits on the LQ mass for all LQ types assuming a coupling of electromagnetic strength λ eq = λ µq (λ τ q ) = 0.3.

Allowing for an arbitrary decay rate between the LFC and LFV decay channels, β LFV , the excluded regions for two LQ types and four mass values in the λ µq 1 −λ eq 1 (a,b) and λ τ q 1 −λ eq 1 (c,d) planes are presented in figure 5.For very low valu s of β LFV (λ eq ≫ λ µq (λ τ q )), the limits on λ eq reproduce the bounds published in [13], as expected, since the LFC channel dominates the LQ width.For β LFV ≫ 0.5 (λ µq (λ τ q ) ≫ λ eq ) the present analysis extends significantly the published limits on λ eq to lower values.The limit without combination in the contact interaction region (where the cross section is proportional to λ eq i λ µ(τ )q j /m 2 LQ ) forms a cross-diagonal straight line following different values of β LFV .The combination in the contact interaction region, e.g.m LQ = 350 GeV, barely strengthens the limit as the virtual effects of the high mass LQ contact interaction at low values of √ ŝ are marginal compared to the ir educible NC and CC DIS background.Fluctuations of the data may even result in a less stringent combined limit.

Figures 6 and 7 display the 95% CL upper limits on the coupling λ µq and λ τ q of all 14 LQ types to a muon-quark pair and a tau-quark pair, respectively, as a function of the LQ mass leading to LFV in ep collisions, assuming λ eq = λ µq (λ τ q ).The limit curves referring to the LQ types S L 0 and SL 1/2 are identical to profiles of the corresponding excluded regions following the value β LFV = 0.5 in figure 5.The limits are most stringent at low LQ masses with values O(10 −3 ) around m LQ = 100 GeV.Corresponding to the steeply falling parton density function for high values of x, the LQ production cross section decreases rapidly and exclusion limits are less stringent towards higher LQ masses.For LQ mass values near the kinematical limit of 319 GeV, the limit corresponding to a resonantly produced LQ turns smoothly into a limit on the virtual effects of both an off-shell s-channel LQ process and a u-channel LQ exchange.At masses m LQ > √ s the two processes contract to an effective four-fermion interaction, where the cross section is proportional to (λ µ(τ )q λ eq /m 2 LQ ) 2 .This feature is visible in the constant increase of the exclusion limit for masses above the ep centre-of-mass energy of √ s = 319 GeV.Due to initial state QED radiation and very low parton densities for masses near √ s the "kink" of the transition region is shifted to somewhat smaller masses of around 290 − 300 GeV.

It is noticeable that the limits on vector LQs are m re stringent compared to those on the scalars, due to the considerably higher cross section and the slightly higher acceptance.In each plot those LQ types that have couplings to both u and d quarks exhibit the best limit.The limits corresponding to LQs coupling to a u quark are more stringent than those corresponding to LQs coupling to the d quark only, as expected from the larger u quark density in the proton.The LQs S L 0 and S R 0 (V L 0 and V R 0 ) differ only by the decay into a neutrino and a quark of the lefthanded LQ.As this decay channel is not covered in the LFV decay channels, the left-handed LQ cannot be as strictly excluded as the right-handed one.This argument applies to the

esonant production where
he analysis is only sensitive to the partial width of the LQ.In the igh mass region the limits for S L 0 and S R 0 (V L 0 and V R 0 ) are similar, as the four-fermion interaction is independent of the decay width.

The limits on λ µ(τ )q = λ eq derived from the virtual effects of a 500 GeV LQ a e transformed into a limit on the value λ µ(τ )q j λ eq i /m 2 LQ and shown in tables 4 and 5 for F = 0 LQs and in tables 6 and 7 for F = 2 LQs.For each LQ type the limit is ca culated for the hypothesis of a process with only the quarks of flavours i and j involved.With respect to quark flavou s, the selection criteria described in sections 4 and 5 are nclusive since no flavour tagging of the hadronic jet is used.These results may be compared with constraints from low energy experiments, based on the non-observation of LFV in muon scattering and rare decays of mesons and leptons [28 .The interpretation in terms of leptoquark exchange and limits on λ µ(τ )q j λ eq i /m 2 LQ [33] are also shown in tables 4, 5, 6 and 7. Bounds of similar magnitude are observed for processes involving e → τ transitions and charm or bottom quarks.In these cases the limits obtained in the present analysis are often superior to those from low energy experiments.

The results on LFV in LQ production are directly comparable with those from the ZEUS experiment [34].Similar limits are obtained.At hadron colliders LQs are mainly produce in pairs independently of the coupling, and therefore searches cannot constrain LFV couplings.Lower mass limits on the second and third generation leptoquarks extend up to 250 GeV and 150 GeV, respectively, depending on the type and the assumed decay branching ratios [35].Similarly, second and third generation leptoquarks are pair produced in e + e − annihilation where typical lower mass bounds reach values of 100 GeV [36].


Conclusion

A search for lepton flavour violation processes induced by l