PgddlmZmZmZmZmZddlZddlmZddl m Z ddl m Z m Z mZddlmZddlmZddlmZmZmZmZdd lmZdd lmZmZdd lmZdd lm Z m!Z!m"Z"m#Z#m$Z$dd l%m&Z&ddl'm(Z(e#jRe*Z+dZ,dZ-Gddej\Z/Gddej\Z0dZ1d;dZ2dejfde4dejffdZ5 dGd2d3eIJJJr/c|j|j}|d|jjzz}|j|S)Ng?)r8floatr+type_as)r,r7outputs r.forwardzGemma2RMSNorm.forwardCsCAGGI&3!2!2!445~~a  r/c^t|jjd|jS)Nz, eps=)tupler+shaper"r,s r. extra_reprzGemma2RMSNorm.extra_reprJs' ))*+6$((<) __name__ __module__ __qualname__intr:r&r8r=rB __classcell__r-s@r.r r :s&5C5e5 K!=r/r c$eZdZfdZdZxZS) Gemma2MLPct|||_|j|_|j|_t j |j|jd|_t j |j|jd|_t j |j|jd|_ t|j|_ yNFbias) r%r&config hidden_sizeintermediate_sizer'Linear gate_projup_proj down_projr hidden_activationact_fnr,rOr-s r.r&zGemma2MLP.__init__Os  !--!'!9!94#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWXV556 r/c|j|j|j||j|z}|Sr$)rUrWrSrT)r,r7rUs r.r=zGemma2MLP.forwardYs6NN4;;t~~a/@#ADLLQRO#ST r/)rCrDrEr&r=rGrHs@r.rJrJNs 7r/rJc|dd|jddzf}|d|jddzdf}tj| |fdS)z*Rotates half the hidden dims of the input..Nr2r1r!)r@r)cat)r7x1x2s r. rotate_halfr_^sZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r/c|j|}|j|}||zt||zz}||zt||zz}||fS)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezer_)qkcossin position_ids unsqueeze_dimq_embedk_embeds r.apply_rotary_pos_embrjesY( -- &C -- &C3w;q>C/0G3w;q>C/0G G r/ hidden_statesn_repreturnc|j\}}}}|dk(r|S|dddddddddfj|||||}|j|||z||S)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) rN)r@expandreshape)rkrlbatchnum_key_value_headsslenhead_dims r. repeat_kvruso 2?1D1D.E h z!!Qa"23::5BUW\^bdlmM  (;e(CT8 TTr/modulequerykeyvalueattention_maskdropoutscalingsoftcapc ||jdz}t||j} t||j} tj|| j dd|z} || |z } tj | } | |z} |#|ddddddd| jdf} | | z} tjj| dtjj|j} tjj| ||j} tj| | } | j ddj!} | | fS) Nr1rr2)r!dtype)ptrainingr)rtrunum_key_value_groupsr)matmul transposetanhr@r' functionalsoftmaxfloat32torr{r contiguous)rvrwrxryrzr{r|r}kwargs key_states value_states attn_weights causal_mask attn_outputs r.eager_attention_forwardrsA//4'3 ; ;$>&B\B\$\!33T9 !%!>!>ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii  & & 68J8JQWQfQf  '+kk&H&H#;? A ;Nf33TXr/rkposition_embeddingsrzpast_key_valuecache_positionrrmc |jdd}g|d|j}|j|j|j dd} |j |j|j dd} |j |j|j dd} |\} } t| | | | \} } |~| | ||jd}|j| | |j|\} } |J|jjdk(r1|jd}| ddddd|ddf| ddddd|ddf} } t}|jjdk7r^|jjdk(r(|jdd rtj!d nt"|jj}||| | | |f|j$r |j&nd |j(|j|j*d |\}}|j,g|dj/}|j1|}||fS) Nr2rr1)rerdrrflash_attention_2eagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.)r{r|rr})r@rtrviewrrrrjrupdaterrO_attn_implementationrgetlogger warning_oncerrrr|rrprr)r,rkrrzrrr input_shape hidden_shape query_statesrrrdre cache_kwargsseq_lenattention_interfacerrs r.r=zGemma2Attention.forwardsa$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &S#7 jRUWZ#[ j  %"0"&"5"5 L (6'<'d##L '>dkk>^>^&_#$7      % /3mmD**LL..// %  % ! \*k));;;;FFH kk+. L((r/)NN)rCrDrE__doc__rrFr&r)Tensorrrr LongTensorrrr=rGrHs@r.rrsGY|YY<+/59 ;)||;)#5<<#=>;)!. ;) ! ;) !!1!12 ;)-.;) u||Xell3XeELL>Q5RR S;)r/rcdeZdZdedeffd Z ddejdeejejfde ejde ejde e d e e d e e d e ejd ed eeje eejejfffdZxZS)Gemma2DecoderLayerrOrct||j|_||_t |dz |_t |||_t||_ t|j|j|_ t|j|j|_ t|j|j|_t|j|j|_|j |_y)Nr1)rOrr")r%r&rPrOr is_slidingr self_attnrJmlpr rms_norm_epsinput_layernormpost_attention_layernormpre_feedforward_layernormpost_feedforward_layernormrrs r.r&zGemma2DecoderLayer.__init__ s !-- "9q=11()LV$,V-?-?VEXEXY(5f6H6HfNaNa(b%)6v7I7IvObOb)c&*78J8JPVPcPc*d'$33r/rkrrzrfrr use_cacherlast_cache_positionrmc |jr|t|jd|j} |jj dk(r |dd| df}nt j|jj} t jt j|t j|j } t j| | |}| | z }td|}|dddddd||| zf}|}|j|}|jd||||||||d| \}}|j!|}||z}|}|j#|}|j%|}|j'|}||z}|f}|r||fz }|S)Nrrrdiagonal)rkrrzrfrrrr)rmaxr@rrOrr)finformintril ones_likerwhererrrrrr)r,rkrrzrfrrrrrreffective_seq_len min_dtypesliding_window_maskoffsetresidualself_attn_weightsoutputss r.r=zGemma2DecoderLayer.forwards ??~9 #N$8$8$;T=P=P Q {{//3FF!/4E3E3F0F!G"KK(<(<=AA &+jjOON%**EQUQdQdPd'#"'-@)^!\-/@@Q!/1a&K\B\9\0\!] ,,]; ,:4>> , ' 3)%)/) ,  , ( (55mD  =0  66}E / 77 F  =0  "  )+ +Gr/)NNNFFNr)rCrDrErrFr&r)rrrrr r FloatTensorr=rGrHs@r.rr s  4| 4 4$2637*.,1$)59#$A||A#5<<#=>A!. A u//0 A ! A$D>AD>A!!1!12A!A u  (51B1BEDUDU1U+V"WW XAr/rcZeZdZddeffd ZdZejdZxZ S)Gemma2RotaryEmbeddingrOct|t|drG|j;|jj d|jj d|_nd|_|j |_|j |_||_ t|j |_ |j|j|\}|_ |jd|d|j|_y)N rope_scaling rope_typetypedefaultinv_freqF persistent)r%r&hasattrrrrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenrOr rope_init_fnattention_scalingregister_bufferroriginal_inv_freq)r,rOdevicerr-s r.r&zGemma2RotaryEmbedding.__init__]s  6> *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+%>v%FD "  T-C-CPU V&*&?&?D # 4g .r/ctd|jvr|j||j|jddddfj j |j ddd}|dddddfj }|jj}t|tr|dk7r|nd}tj|d 5|j j|j|j zjdd }tj||fd }|j}|j!} ddd|j"z} |j"z} |j|j$ | j|j$ fS#1swY_xYw) Ndynamicrrr2rmpscpuF) device_typeenabledr1r[r)rrrrr:ror@r isinstancestrr)autocastrrr\rdrerr) r,r7rfinv_freq_expandedposition_ids_expandedrfreqsembrdres r.r=zGemma2RotaryEmbedding.forwardsu  &  * *< * I!MM$4-8>>@GG HZHZ[\H]_acde ,QaZ 8 > > @hhmm %/ S%AkUZFZk`e ^^ U C &,,.11!((;>S>Y>Y>[[ffghjklE))UEN3C'')C'')C  D***D***vvAGGv$cff177f&;;;  s BF..F7r$) rCrDrErr&rr)no_gradr=rGrHs@r.rr\s1/|/"@&U]]_<..r/N)rCrDrEr config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_attention_backendrrr/r.rrsU  L&*#-.#4"5!N  $!"& ?r/ra Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. If `past_key_values` is used, optionally only the last `input_ids` have to be input (see `past_key_values`). If you want to change padding behavior, you should read [`modeling_opt._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_values (`Cache` or `tuple(tuple(torch.FloatTensor))`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. Two formats are allowed: - a [`~cache_utils.Cache`] instance, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache); - Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`). This is also known as the legacy cache format. The model will output the same cache format that is fed as input. If no `past_key_values` are passed, the legacy cache format will be returned. If `past_key_values` are used, the user can optionally input only the last `input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length. cLeZdZdZdeffd ZdZdZee  dde jde e jde e jd e ed e e jd e ed e ed e ede ede e jde edeedeeeffdZe j0de jde jde jd ed ef dZede jdedede j6de j8de jdefdZxZS) Gemma2Modelz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`Gemma2DecoderLayer`] Args: config: Gemma2Config rOc  t|||j|_|j|_t j |j|j|j|_t jt|jDcgc]}t||c}|_ t|j|j|_t#||_d|_|j)ycc}w)Nr)rOF)r%r& pad_token_idr vocab_sizer'r rP embed_tokens ModuleListrangenum_hidden_layersrlayersr rnormr rotary_embgradient_checkpointing post_initrs r.r&zGemma2Model.__init__s  !.. ++LL):):F&+#  esDc|jSr$r!rAs r.get_input_embeddingsz Gemma2Model.get_input_embeddings/s   r/c||_yr$r+r,rys r.set_input_embeddingsz Gemma2Model.set_input_embeddings2s !r/ input_idsrzrfr inputs_embedsrroutput_hidden_states return_dictrrflash_attn_kwargsrmc ||n|jj}||n|jj}||n|jj}| | n|jj} |du|duz r t d|j r%|jr|rtjdd}||j|}|rL|J|js>|j\} }}t|j| ||j|j}| F||jnd}t!j"|||jdz|j} || j%d}| 9d} |5|j'dk(r|jd n| d j)} |j+||| ||}|}|j-||}t!j.|jj0d z|j }||z}|rd nd}|rd nd}|j2d|jj4D]j}|r||fz }|j r1|jr%|j7|j8|||||||| | }n||f||||||| | d | }|d}|sb||dfz }l|j;|}|r||fz }t=||||}| r|S|j?S)Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.F)max_batch_size max_cache_lenrrrrrr1r2g?rr)rrzrfrrrrr)last_hidden_staterrk attentions) rOrr2ruse_return_dict ValueErrorr(rrrr!r@r rrget_seq_lengthr)arangerar!item_update_causal_maskr'tensorrPr%r$_gradient_checkpointing_func__call__r&rto_tuple)r,r0rzrfrr1rrr2r3rrr4 batch_sizer_past_seen_tokensrrkr normalizerall_hidden_statesall_self_attns decoder_layer layer_outputsr<s r.r=zGemma2Model.forward5sW 2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?L] & #oom\J \\$++"9"93">mFYFYZ % 2 #7BD0d![[)H4;;+H+HI" 6M#!m%55!**t}} $ A A!**!' #%"' ! !.! !(;#.!-#2&7'#1(; !( ! *!,M =#3"55E" 6H -0  -!1 1 (+++%   %v;&//*;;r/ input_tensorc b|jjdk(r|S|j|j}}|jd}t |t tfr|j} n ||jdn|jd} |j||| ||||jd} | S)Nrrr2rsequence_length target_lengthrrrrD) rOrrrr@rr r get_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_position) r,rzrLrrrrrrOrPrs r.r?zGemma2Model._update_causal_masks ;; + +/B B! !$**L,?,?v&,,Q/ o ['A B+??AM8F8RN004XdXjXjklXmMPP +')#))!,Q r/rOrPrrrDc ||jdk(r|}|Stj|j} tj||f| ||}|dk7rtj |d}|tj |||jddkDz}|ddddddfj|ddd}||j}|jd} |ddddddd| f|ddddddfj|jz} | dk(} |ddddddd| fj| | |ddddddd| f<|S) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. device (`torch.device`): The device to place the 4D attention mask on. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerrrrrr2r)r!r)rrfulltriur=rprocloner@rr masked_fill) rzrOrPrrrrDrrr mask_length padding_masks r.rRzAGemma2Model._prepare_4d_causal_attention_mask_with_cache_positionsyB  %.*<*<*>!*C(K*' E*..I** -0Ye\bK!##jjqA 5<< fEH^H^_acdHee eK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c )6 Aq!\k\12r/) NNNNNNNNNNN)rCrDrErrr&r,r/rGEMMA2_INPUTS_DOCSTRINGr)rrrr rrrFrrrrrr=rr? staticmethodrrrRrGrHs@r.rrs | !"++BC'+15371559$(,0/3&*59-1C<##C<!.C<u//0 C< "+. C<   1 12 C<D>C<$D>C<'tnC<d^C<!!1!12C<&c]C<$$89C< u-- .C<DC>> from transformers import AutoTokenizer, Gemma2ForCausalLM >>> model = Gemma2ForCausalLM.from_pretrained("google/gemma-2-9b") >>> tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b") >>> prompt = "What is your favorite condiment?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "What is your favorite condiment?" ```rzhIt is strongly recommended to train Gemma2 models with the `eager` attention implementation instead of `zp`. Use `eager` with `AutoModelForCausalLM.from_pretrained('', attn_implementation='eager')`.N) r0rzrfrr1rrr2r3rrrlossrbrrkr9r)rrOrrrrr2r:rrrFslicer`final_logit_softcappingr)r loss_functionr rrrkr9)r,r0rzrfrr1rzrrr2r3rrv loss_kwargsrrk slice_indicesrbr}r<s r.r=zGemma2ForCausalLM.forward9sf ==T[[==H   #{{??@Aqr 2C1N-TXT_T_TqTq$8$D $++JjJj &1%SV8W~ot4]k mA}a,?@A ;; . . :dkkAAAFZZ'FdkkAAAF  %4%%ffdooUUDY,F'+'7D7V# CV C%#33!//))   r/c 4t||f|||||||d| } ||jdnd| d<|| jdd} t |t r|j dk(r|jjdk(s| d#| dj\} } } | dj}n!| d j\} } | d j}|jj|| |j|jjj||| }|| d <| S) N)rrzr1rrfrrvr2rrrvr1rr1r0rNrz)r%prepare_inputs_for_generationr@poprr ndimrOrrrrRrQr`r+r)r,r0rrzr1rrfrrvr model_inputsrErDrOrr-s r.rz/Gemma2ForCausalLM.prepare_inputs_for_generationsUw<   +)')%)    KYJdn.B.B2.Fjk *+  !  !148A  4##q(KK448KKO,81=o1N1T1T. OQ%o6==.:;.G.M.M+ O%k299!ZZ]] /-AACll))//-%^N.` $D>` 'tn` d^` !!1!12` c5<</0`  u,, -` aDV` J77r/r_a The Gemma2 Model transformer with a sequence classification head on top (linear layer). [`Gemma2ForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). cXeZdZfdZdZdZee ddee jdee jdee jdee e ee jfdee jd ee jd eed eed eed eede eeffdZxZS)Gemma2ForSequenceClassificationct|||j|_t||_t j |j|jd|_|jyrL) r%r& num_labelsrrr'rRrPscorer)rXs r.r&z(Gemma2ForSequenceClassification.__init__sS   ++ ( YYv114??O  r/c.|jjSr$rerAs r.r,z4Gemma2ForSequenceClassification.get_input_embeddingsrfr/c&||j_yr$rer.s r.r/z4Gemma2ForSequenceClassification.set_input_embeddingsrhr/r0rzrfrr1rzrrr2r3rmc | | n|jj} |j|||||||| |  } | d} |j| } ||jd}n|jd}|jj |dk7r t d|jj d}n|||jj k7j| jtj}tj|jd| jtj}||zjd}n.d}tj|jj d| tj|| j |f}d}||j#| |||j }| s|f| ddz}||f|zS|St%||| j&| j(| j* S)  labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Nrzrfrr1rrr2r3rrz=Cannot handle batch sizes > 1 if no padding token is defined.r2)rrz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r)rbrz pooled_logitsrOr|)rOr:rrr@rr;rrr)int32r=argmaxrrr-rCrrrrkr9)r,r0rzrfrr1rzrrr2r3transformer_outputsrkrbrDlast_non_pad_token non_pad_mask token_indicesrr}r<s r.r=z'Gemma2ForSequenceClassification.forwards(&1%!F!Fr!J !#    >>**+,ZZ  u||Jv}}MOaab   %%VFR_hlhshs%tD#%(;AB(??F)-)9TGf$ Ev E/ /??-;;*55   r/ NNNNNNNNNN)rCrDrEr&r,r/rr\rr)rrrr rrrrrr=rGrHs@r.rrs0 '(++BC151537KO59-1$(,0/3&*H E,,-H !.H u//0 H "%tE4E4E/F(F"GH H   1 12 H ))*H D>H $D>H 'tnH d^H  u66 7H DH r/rz The Gemma2 Model transformer with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for Named-Entity-Recognition (NER) tasks. cjeZdZfdZdZdZeeee e e  dde e jde e jde e jde ee j"d e e j"d e e jd e ed e ed e ede edeee ffdZxZS)Gemma2ForTokenClassificationct|||j|_t||_t |dd |j }nt |dd |j}nd}tj||_ tj|j|j|_ |jy)Nclassifier_dropouthidden_dropoutg?)r%r&rrrrrrr'Dropoutr{rRrPrr))r,rOrr-s r.r&z%Gemma2ForTokenClassification.__init__Ks   ++ ( 6/ 6 B!'!:!:  V-t 4 @!'!6!6 !$ zz"45 YYv1163D3DE  r/c.|jjSr$rerAs r.r,z1Gemma2ForTokenClassification.get_input_embeddings[rfr/c&||j_yr$rer.s r.r/z1Gemma2ForTokenClassification.set_input_embeddings^rhr/) checkpointryrr0rzrfrr1rzrrr2r3rmc j| | n|jj} |j|||||||| |  } | d} |j| } |j | } d}||j | ||j}| s| f| ddz}||f|zS|St || | j| jS)rNrrr1)r}rbrkr9) rOr:rr{rrrrkr9)r,r0rzrfrr1rzrrr2r3rsequence_outputrbr}r<s r.r=z$Gemma2ForTokenClassification.forwardas2&1%1 $D>1 'tn1 d^1  u++ ,1 D 1 r/r)r_rrrr)Nr)rNN)Ctypingrrrrrr)torch.nnr' activationsr cache_utilsr r r generationr modeling_flash_attention_utilsrmodeling_outputsrrrrmodeling_rope_utilsrmodeling_utilsrrprocessing_utilsrutilsrrrrrutils.deprecationrconfiguration_gemma2r get_loggerrCrrrModuler rJr_rjrrFrur:rrrrGEMMA2_START_DOCSTRINGrr\rr_rr__all__rr/r.rse,:9 !::)B 7F&1.   H %) =BII=(   (6 UU\\ U# U%,, U$## % II % << %  % << % U\\* %  %e_ %e_ % 5<< %& %FW)biiW)tPPf: