????

Your IP : 3.129.70.104


Current Path : /opt/imunify360/venv/lib/python3.11/site-packages/im360/model/__pycache__/
Upload File :
Current File : //opt/imunify360/venv/lib/python3.11/site-packages/im360/model/__pycache__/firewall.cpython-311.pyc

�

��g	���dZddlZddlZddlZddlZddlmZddlmZddl	m
Z
ddlmZmZddlm
Z
ddlmZmZdd	lmZmZmZmZmZmZmZmZmZmZmZdd
lmZddlm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/ddl0m1Z1dd
l2m3Z3ddl4m5Z5m6Z6ddl7m8Z8ddl9m:Z:m;Z;m<Z<ddl=m>Z>ddl?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFddlGmHZHmIZImJZJeKZLejMeN��ZOeDed����dZPeDed����dZQeJeHjRjSZTeJeHjUjSZVdee5deeKeKeKffd�ZWdee5deeKeKeKffd�ZXdee5deeKeKeKffd�ZYd�ZZd�Z[d�Z\d�Z]d e^fd!�Z_Gd"�d#e��Z`Gd$�d%eae��ZbGd&�d'e5��ZcGd(�d)e5��ZdGd*�d+e5��ZeGd,�d-e5��ZfGd.�d/e5��ZgGd0�d1e5��ZhGd2�d3e5��ZiGd4�d5e5��ZjGd6�d7e5��ZkGd8�d9e5��ZlGd:�d;e5��ZmGd<�d=e5��ZndS)>z,DB tables related to firewall functionality.�N)�	timedelta)�Enum)�reduce)�IPv4Network�IPv6Network)�starmap)�ior�
itemgetter)�Any�Dict�Iterable�Iterator�List�Optional�Sequence�Tuple�Type�Union�Set)�Signal)�JOIN�SQL�BooleanField�Case�	CharField�Check�CompositeKey�DoesNotExist�
FloatField�ForeignKeyField�IntegerField�PrimaryKeyField�	TextField�fn�prefetch�Field)�
model_to_dict)�Reject)�Model�instance)�ApplyOrderBy)�CHUNK_SIZE_SQL_QUERY�split_for_chunk�timeit)�Country)�ALL�TCP�UDP�	IPNetwork�pack_ip_network�unpack_ip_network�is_net)�IP�	IPVersion�NumericIPVersionz
0.0.0.0/32�z::/64�model�
packed_ip_netc�b�|\}}}td||f��|j|kz|j|kzS)z�
    Filters ip addresses/networks contained in ip network net.

    :param model: model to apply query
    :param packed_ip_net: tuple of integers
    :return: peewee expression
    �(network_address & ?) == ?�r�netmask�version�r;r<�net�maskrAs     �I/opt/imunify360/venv/lib/python3.11/site-packages/im360/model/firewall.py�_filter_ip_net_subnetsrFJsE��'��C��w��(�4��+�6�6��=�D� �	"��=�G�#�	%��c�b�|\}}}td||f��|j|kz|j|kzS)z�
    Filters ip addresses/networks contained in ip network net.
    Does not includes network itself
    :param model: model to apply query
    :param packed_ip_net: tuple of integers
    :return: peewee expression
    r>r?rBs     rE� _filter_ip_net_subnets_exclusiverI\sE��'��C��w��(�4��+�6�6��=�4��	!��=�G�#�	%�rGc�`�|\}}}td|f��|j|kz|j|kzS)z�
    Filters ip addresses/networks that includes provided
    ip address/network, including network itself
    :param model: model to apply query
    :param packed_ip_net: tuple of integers
    :return: peewee expression
    z (? & netmask) == network_addressr?rBs     rE�_filter_ip_net_supernetsrKnsC��'��C��w��.���7�7��=�D� �	"��=�G�#�	%�rGc��	tj|��}t|��\}}}|js.|j�|��t
||||f��zS|j�|��t
||||f��zt||||f��zS#t$r|j�|��cYSwxYw�N)	�	ipaddress�
ip_networkr4�hostmask�ip�containsrKrI�
ValueError)r;�ip_str�ip_netrCrDrAs      rE�_ip_search_conditionrV�s�����%�f�-�-��-�V�4�4���T�7���		��8�$�$�V�,�,�/G���T�7�+�0�0��
�
��!�!�&�)�)�*�5�3��g�2F�G�G�H�2�5�3��g�:N�O�O�P�
���)�)�)��x� � ��(�(�(�(�(�)���s�B � $C�Cc�v�t|��\}}}t||||f��t||||f��zSrM)r4rKrI)r;rUrCrDrAs     rE�_net_search_conditionrX�sO��(��0�0��C��w�#�
��T�7�#���(���d�G�0D�E�E�F�FrGc�v��t�fd�dD����r;t�d�d�d��}tj|���d<nad�vr]tj�d��}t|��\}}}��|||d���tj|���d<�S)Nc3� �K�|]}|�vV��	dSrM�)�.0�k�kwargss  �rE�	<genexpr>z#_add_ip_net_args.<locals>.<genexpr>�s'�����
J�
J�1�1��;�
J�
J�
J�
J�
J�
JrG)�network_addressr@rAr`r@rArQ)�allr5r7�ip_net_to_string�adopt_to_ipvX_networkr4�update)r^rQrCrDrAs`    rE�_add_ip_net_argsre�s����
�
J�
J�
J�
J� I�
J�
J�
J�J�J�/�
��$�%�v�i�'8�&��:K�
�
���*�2�.�.��t���	
����
�
%�f�T�l�
3�
3��,�R�0�0���T�7��
�
� #���I�I�	
�	
�	
��*�2�.�.��t���MrGc��t|�d��ttf��r%t	|d��\|d<|d<|d<|d=|S)NrQr`r@rA)�
isinstance�getrrr4)�argss rE�_replace_ip_with_packed_reprrj�s\���$�(�(�4�.�.�;��"<�=�=��

�D��J�'�'�		
��"�#���O���O���J��KrG�returnc�P�|tjko|tj��kS)zWhether expiration time passed.)�IPList�NEVER�time��
expirations rE�
is_expiredrr�s�����%�C�*��	���*C�CrGc�"�eZdZdZdZdZdZdZdS)�
ActionTypezWhat to do with matching IPs.�drop�captcha�splashscreen�ignoreN)�__name__�
__module__�__qualname__�__doc__�DROP�CAPTCHA�SPLASHSCREEN�IGNOREr[rGrErtrt�s*������'�'��D��G�!�L�
�F�F�FrGrtc�T�eZdZdZdZeZdZeZdZeZ	dZ
e
Zd�Ze
defd���Zd	S)
�Purposez�IPList's purposes understood by the agent.

    An analog of i360.model.firewall.ActionType but for the new
    (DEF-17989) server sync case.

    �whiterurwrvc��|jSrM)�value��selfs rE�__str__zPurpose.__str__�s
���z�rG�purposec���tjtjtjtjtjtjtjtjit|SrM)	r��WHITErmr}�BLACKr~�GRAYr�GRAY_SPLASHSCREEN)�clsr�s  rE�listnamezPurpose.listname�sC��
�M�6�<��L�&�,��O�V�[�� �&�":�	
�
�'�
��	rGN)ryrzr{r|r�r�r}rurrwr~rvr��classmethod�strr�r[rGrEr�r��s~��������
�E��E��D��D�!�L��L��G��G������s�����[���rGr�c�
��eZdZdZdZdZdZdZdZeeeefZ	e
eee	������Z
dZdZd	ZeZeZd
ZGd�d��Zed
���Zed
ed�d�e	������g���ZdZedd���Zed���Z edd����Z!ed���Z"ed���Z#edd���Z$e%d
d
���Z&e%d
d���Z'e%d���Z(e%dd
���Z)ed
���Z*ed
���Z+ed
���Z,edede�de�d���g���Z-Gd�d��Z.Gd�d��Z/e0de1e2de2fd ���Z3e0d!e2de2fd"���Z4e0d#e1e5fd$���Z6e0d#e1e5de7fd%���Z8e9de:e2e7ffd&���Z;e9		dbd'e<d(e=e2d)e2d*e7d+e>f
d,���Z?e9�fd-���Z@e9	dcd'eAe2eBeCfd!e=e2d/e>fd0���ZDe9d1���ZEe9d2���ZFe9		ddd!e2d'e<fd3���ZGe9						ded4���ZHe9d5e=e2fd6���ZIe9d7e=e<fd8���ZJe9d.d.d.eK��fd5e=e2d9e1e7d:e1e7d;e1e/d<eLeMf
d=���ZNe9dcd>���ZOe9ddd?���ZPe9ddd@���ZQe9ed.fdeRe<fdA���ZSe9dB���ZTe9dC���ZUe9dcdD���ZVdE�ZWe9dF���ZXdG�ZYdH�ZZdcdI�Z[e9d'e<de1e2fdJ���Z\e9dK���Z]e9dL���Z^e_dM���Z`e9		ddd.d.dN�d'eAeajBeajCfd!e=e2d:e7fdO���Zbe9dP���Zce9		dddQede<ede2eeffd5e1e=fdR���Zfe9d7ede<ede2eefffdS���Zge9d7ede<ede2eefffdT���Zhe9d.d.dU�d'e<d!e=e2fdV���Zie9�fdW���Zje9�fdX���Zke9�fdY���Zle9	dcd
d.d.dZ�d'e<d!e=e2d/e>de=e:e<e2e7ffd[���Zme9d!e2dene2fd\���Zoe9d!e2dene2fd]���Zpe9dcd'e<d!e2fd^���Zqe9d_e=edfd`���Zre9d7eRe:e<edfde=e:e<e2ffda���Zs�xZtS)frmz1The main persistent storage for various IP lists.�action_typer�r�r�r��local�groupr��2c�b�eZdZdZe��Ze��Ze��Ze��ZdS)�IPList.Signalsz[Signals to inform subscribers about changes.

        Sender of event is listname.
        N)	ryrzr{r|r�added�deleted�cleared�updatedr[rGrE�Signalsr�sL������	�	�������&�(�(���&�(�(���&�(�(���rGr�F��nullzlistname in ('{}')z','�r��constraintsrT��defaultr�c�B�ttj����SrM)�intror[rGrE�<lambda>zIPList.<lambda>2s��3�t�y�{�{�#3�#3�rG�r�r��
country_id�r��column_namezscope in ('�')c�@�eZdZejZdZedddd��ZdZ	dS)�IPList.Meta�iplistr`r@rAr��residentN�
ryrzr{r*�db�database�db_tabler�primary_key�schemar[rGrE�Metar�[s;�������;����"�l��y�)�Z�
�
�����rGr�c�$�eZdZed���ZdS)�IPList.OrderByc�P�ttjdd��tjfS)N))rr:r)rrmrqr[rGrErqzIPList.OrderBy.expirationds����)�9�a�8�8�&�:K�K�KrGN)ryrzr{�staticmethodrqr[rGrE�OrderByr�cs2������	�	L�	L�
��	L�	L�	LrGr�rkc
�2�tjjtjtjjtjtjjtjtj	jtj	dtji�
|tj��S)z�Given `action_type` string return corresponding list name.

        Return :attr:`GRAY` for an unknown/missing `action_type`.
        N)rtr}r�rmr�r~r�rr�r�rh)r�s rE�action_type2listnamezIPList.action_type2listnamehs\��
�O�!�6�<���$�f�k��#�)�6�+C���#�V�]��&�+�
��#�k�6�;�
'�
'�
	(rGr�c��tjtjjtjtjjtjtjji|S)z0Return action_type corresponding to iplist name.)	rmr�rtr}r�r�r~r�r�r�s rE�listname2action_typezIPList.listname2action_typevs?��
�L�*�/�/��K��+�1��$�j�&=�&C�
��	�	rG�
propertiesc�x�t�|�|�tj��nd��S)zzGet iplist name corresponding to properties' action_type.

        Return GRAY for an unknown/missing action_type
        N)rmr�rh�ACTION_TYPE�r�s rE�get_listname_fromzIPList.get_listname_froms<���*�*��%�
�N�N�6�-�.�.�.��
�
�	
rGc�^�|r |�dtj��ntjS)zSGet expiration from properties

        Return IPList.NEVER if property not definedrq)rhrmrnr�s rE�get_expiration_fromzIPList.get_expiration_from�s+���
�J�N�N�<���6�6�6���	
rGc�r�|�|��}|jD]\}}||kr||fcS�Jd���)z#Return tuple listname and priority.rzcan't happen)r��IP_LIST_PRIORITIES)r�r��	_listname�priorityr�s     rE�get_listname_with_priority_fromz&IPList.get_listname_with_priority_from�s\���)�)�*�5�5�	�"%�"8�	*�	*��H�h��9�$�$���)�)�)�)�%� �.� � �qrGrQ�src�destrq�full_accessc���||j|jfvs
Jd���t|��\}}}d�t�tjtj�����tj	�
|��tj|ktj|ktj
|k��D��\}	t����tj	�
|��tj|ktj|ktj
|ktj|	k�����t�|||dd����tj	�
|����}
|
�tj|ktj|ktj
|k��}
|
���}|D]#}|jj�||����$|jj�||�|������|S)aMove ip from src lists to dest list,
        as `move` used only in UI and CLI we add manual=True

        :param ip: ip address
        :param src: src lists (WHITE/BLACK/GRAY/GRAY_SPLASHSCREEN)
        :param dest: dst list (WHITE/BLACK)
        :param expiration: IPs TTL. 0 means permanent
        :param full_access: access to all ports
        :return int: items moved
        z"Move to GRAY list is not supportedc��g|]	}|j��
Sr[rp)r\�recs  rE�
<listcomp>zIPList.move.<locals>.<listcomp>�s*��
�
�
��
�N�
�
�
rGTF)r�rqr��manual�captcha_passed�rQ)r�r�r4rm�selectr$�MAXrq�wherer��in_r`r@rA�delete�executerdr�r��sendr�rh)
r�rQr�r�rqr�rCrDrA�max_expiration�q�rvr�s
             rE�movezIPList.move�s��(��H��!�
�
�
�
�0�
�
�
�
-�R�0�0���T�7�
�
��}�}�R�V�F�,=�%>�%>�?�?�E�E���#�#�C�(�(��&�#�-���$�&���'�)�	��
�
�
���	�
�
������O����$�$��"�c�)��N�d�"��N�g�%����/�	
�	
��'�)�)�)��M�M��!�#�� �
�
�
��%���#�#�C�(�(�
)�
)�
	
�
�G�G��"�c�)��N�d�"��N�g�%�
�
��
�Y�Y�[�[���	6�	6�H��K��$�$�X�"�$�5�5�5�5������t����2�����7�7�7��	rGc���tt|��jdit|����}|jj�|j|���|S)�`
        :param kwargs:
        :raises: IntegrityError
        :return: model instance
        r�r[)�superrm�createrer�r�r�r��r�r^�inst�	__class__s   �rEr�z
IPList.create�sU���)�u�V�S�!�!�(�D�D�+;�F�+C�+C�D�D�������t�}���6�6�6��rGNr�c��|�td���t����tj�|����}|�|�|j|k��}t|t��r$|�tj	|k��}nTt|��\}}}|�tj|ktj|ktj
|k��}|���}|r&|D]#}	|jj�|	|����$|S)zQDelete ip from lists if exists
        Return number of deleted records.
        Nzlistname should not be Noner�)rSrmr�r�r�r�r�rgr�rQr4r`r@rAr�r�r�r�)
r�rQr�r�r�rCrD�ver�rows_deleted�lsts
          rE�delete_from_listzIPList.delete_from_list�s�����:�;�;�;��M�M�O�O�!�!�&�/�"5�"5�h�"?�"?�@�@��������
�f�,�-�-�A��b�#���	�����	�R��(�(�A�A�,�R�0�0�N�C��s�����&�#�-���$�&���#�%���A�
�y�y�{�{���	5��
5�
5����#�(�(���(�4�4�4�4��rGc��t����tj�|�������}|r$|D]!}|jj�|���"|SrM)	rmr�r�r�r�r�r�r�r�)r��	listnames�num_deletedr�s    rE�clean_listszIPList.clean_listssy��
�M�M�O�O�!�!�&�/�"5�"5�i�"@�"@�A�A�I�I�K�K�	��	3�%�
3�
3����#�(�(��2�2�2�2��rGc���tj��t|������z
}|j|jk|�|��z}|j|jk|�|��z}|j|jk|jdkz|�|��z}|�	���
||z|z�����}|S)z�
        Removes obsoleted graylist/splashscreen+blacklist[manual=False] IPs.
        :param num_days: expired more than num_days ago
        :return: int rows deleted
        )�daysF)ror�
total_secondsr�r�rrr�r�r�r�r�r�)r��num_days�
expiration_ts�graylist_ip_is_expired�graysplash_ip_is_expired�blacklist_ip_is_expiredr�s       rE�cleanup_expired_from_bglistz"IPList.cleanup_expired_from_bglists����	���i�X�&>�&>�&>�&L�&L�&N�&N�N�
�"%�,�#�(�":�c�n�n��?
�?
�"
��

�L�C�1�1��N�N�=�)�)�$*� �
�\�S�Y�
&��z�U�"�
$��n�n�]�+�+�
,�	 �
�J�J�L�L�
�U�&�*�+�)�*���
�W�Y�Y�	��rGc�T�|���}|r||j|kz}|rHt|��\}}}|tj|ktj|kztj|kzz}|����|���	��}|SrM)
rrr�r4rmr`r@rAr�r�r�)r�r�rQ�clausesrCrDrAr�s        rE�delete_expiredzIPList.delete_expired3s����.�.�"�"���	0��s�|�x�/�/�G�
�	�!0��!4�!4��C��w���'�3�.��>�T�)�+��>�W�,�.�
�G��*�*�,�,�$�$�W�-�-�5�5�7�7���rGc�n��t|t��sJ�|��fd�|D��}t�tjtjtjtjtjtj	tj
tjtjtj
tjtjtjtjt%jtj�j���d�����t0t2jtjt0jk����tj�|�����t������tj��}|�|� |��}|�|�!|��}|�|��j|k��}|rEtE|��}	|	r	tF|	fntH|f\}
}	|�|
�|	����}|r#|�t0j%|k��}|�2|�tj
�&|����}|S)Nc�4��g|]}|�j�jfv�|��Sr[)r�r�)r\�lnr�s  �rEr�z'IPList._fetch_query.<locals>.<listcomp>Vs8��������c�h��(=�>�>�>��>�>�>rG�scope��on)'rg�listrmr�rQr�rq�
imported_from�ctime�deep�comment�countryr�r��auto_whitelistedr`r@rAr$�ifnullr�SCOPE_LOCAL�alias�joinr/r�
LEFT_OUTER�idr�r�rr�order_by�group_by�havingr6rXrV�coderR)r�r�rr�by_ip�by_country_code�
by_commentr�r�rC�search_conditions`          rE�_fetch_queryzIPList._fetch_queryGs,����)�T�*�*�*�*�*��!�����#����I�
�M�M��	����!��$����������
��"��'��&������	�&�,���8�8�>�>�w�G�G�
�
�"�T�'�4�?���'�*�0L�T�
N�
N�
�U�6�?�&�&�y�1�1�
2�
2�
�U�F�%�%�'�'�'�
(�
(�
�X�f�i�
 �
 �+	
�2���
�
�8�$�$�A������� � �A�������
�f�,�-�-�A��	4���-�-�C��3�&��,�,�*�E�2�
"��c�
���(�(��c�2�2�3�3�A��	9�������7�8�8�A��!������/�/�
�;�;�<�<�A��rGr�c�B�|j|fi|�����SrM�r�count)r�r��filter_argss   rE�fetch_countzIPList.fetch_count�s)���s��	�9�9�[�9�9�?�?�A�A�ArG�ipsc��t����tjtjk��}g}t|t
tdz����D]�}g}|D]Z}t|��\}}}	|�	tj
|ktj|kztj|	kz���[tt|��}
|t|�|
����z
}��|S)z�
        cannot use 'in' operator, since peewee's @hybrid_property can not
        compute lazy expression for 'IPList.ip_network' property
        �)rmr�r�r�SCOPE_GROUPr-r�r,r4�appendr`r@rArr	r)r�r r��result�chunk�expressionsrQrCrDrA�clauses           rE�fetch_for_group_synczIPList.fetch_for_group_sync�s���
�M�M�O�O�!�!��L�F�.�.�
�
����$�S�#�.B�Q�.F�*G�*G�H�H�
	,�
	,�E��K��
�
��%4�R�%8�%8�"��T�7��"�"��+�s�2��~��-�/��~��0�2�����
�C��-�-�F��d�1�7�7�6�?�?�+�+�+�F�F��
rG�offset�limitr�exclude_fieldsc�"�|j|fi|��}|�|�|��}|�|�|��}|��d�|D��}d�|D��}	g}
|D]V}|
�|jr|���n%|���������W|	D]d}t
j||j�	d����}|D]2}
|
�|jr|
���n|
���3�e|j
|
�}g}|D]g}t||���}|�d��r*ttj|j�����|d<|�|���h|S)z:
        :return tuple: (max count, list of dict)
        Nc�(�g|]}|jdk�
|��S�r��r��r\�orders  rEr�z IPList.fetch.<locals>.<listcomp>�s,������u�/@�I�/M�/M��/M�/M�/MrGc�(�g|]}|jdk�
|��Sr/r0r1s  rEr�z IPList.fetch.<locals>.<listcomp>�s,������u�/@�I�/M�/M��/M�/M�/MrG�.)�excluder
)r)rr*r+r$�desc�
list_priorityr+�	get_nodesr��splitrr'rhr/r
)r�r�r*r+rr,rr��
purpose_order�others_order�ordersr2�nodes�node�rows�row�entrys                 rE�fetchzIPList.fetch�s���
�C��Y�6�6�+�6�6�������� � �A���������A�����#+����M���#+����L��F�'�
�
���
�
��z�4�C�%�%�'�'�'��*�*�,�,�1�1�3�3�����
&�
G�
G��$�.���*�0�0��5�5����"�G�G�D��M�M���"E�$�)�)�+�+�+��F�F�F�F�G���
�F�#�A����	�	�C�!�#�~�>�>�>�E��y�y��#�#�
N�#0�����1L�1L�1L�#M�#M��i� ��K�K�������rGc�v�|s
Jd���	t|jdi|��|��S#t$r|cYSwxYw)z;Return matching row's field value or `default` if not foundzprovides kwargs to find by themNr[)�getattrrhr)r��fieldr�r^s    rE�	get_fieldzIPList.get_field�sc���8�8�8�8�8�v�	��7�3�7�,�,�V�,�,�e�4�4�4���	�	�	��N�N�N�	���s�)�8�8c�j�|�|j|j���|���|j|kz��}|�>|j|k}|s||j���z}|�|��}|�|�|j|k��}|SrM)	r�rQrqr�rrr�r��is_nullrA)r�r�r�rAr�r(s      rE�fetch_non_expired_queryzIPList.fetch_non_expired_query�s����J�J�s�v�s�~�.�.�4�4��n�n���
�3�<�8�#;�<�
�
���"��_��3�F��
4��#�/�1�1�3�3�3��������A��������w�.�/�/�A��rGc#�K�|�|||��}	|������Ed{V��dS#t$rYdSwxYwrM)rI�dicts�iterator�RuntimeError)r�r�r�rAr�s     rE�fetch_non_expiredzIPList.fetch_non_expired�sv�����'�'��+�w�G�G��	��w�w�y�y�)�)�+�+�+�+�+�+�+�+�+�+�+���	�	�	��F�F�	���s�,A	�	
A�Ac#�K�|�tjtjtj��}|�tj|kt���z��}|�|�|j|k��}|�	��D]'}t|d|d|d��V��(dS)zh
        Fetch listname the most efficient (though experiementally found)
        way possible.
        Nr`r@rA)r�rmr`r@rAr�r�rrr�rKr5)r�r�r�r�r@s     rE�fetch_ipnetwork_listzIPList.fetch_ipnetwork_list�s�����
�J�J�v�-�v�~�v�~�N�N��
�G�G�V�_��0�V�5F�5F�5H�5H�4H�H�I�I��������
�f�,�-�-�A��7�7�9�9�	�	�C�#��%�&��I���I����
�
�
�
�	�	rGc�f�	t�|���jS#t$rYdSwxYw)Nr�)rmrhr�r)r�rQs  rE�get_listnamezIPList.get_listnames@��	��:�:��:�$�$�-�-���	�	�	��D�D�	���s�"�
0�0c�0�|jdk|jdz	zS)�*The result has to be passed to .where(...)rNrp�r�s rE�is_expirablezIPList.is_expirables ����!�#���4�)?�'@�@�@rGc���|tjkr|���S|���|jt	|ptj����kzS)rT)rmrnrVrqr�ro)r�r�s  rErrzIPList.is_expiredsY���F�L�(�(��#�#�%�%�%����!�!��N�c�-�">�4�9�;�;�?�?�?�
�	
rGc�b�|tjko|jtjkp
|j|kS)z@Whether the ip record lives longer than given *expiration* time.)rmrnrq�r�rqs  rE�lives_longerzIPList.lives_longer!s.���V�\�)�
��O�v�|�+�K�t���/K�	
rGc��|tjkoJ|rG|�dtj��tjkp|�d��|kndS)zZWhether the properties for ip lives longer than given *expiration*
        time.

        rqT)rmrnrh)r�r�rqs   rE�lives_longer_propzIPList.lives_longer_prop'sX���V�\�)�
��
�J�N�N�<���6�6�&�,�F�
9��~�~�l�+�+�j�8���		
rGc�b�|jtjko|tjkp
|j|kS)z>Whether the ip record lives less than given *expiration* time.)rqrmrnrYs  rE�
lives_lesszIPList.lives_less4s.����&�,�.�
��&�,�&�F�$�/�J�*F�	
rGc��|j}t|||��}|j|jko|j|jko|j|jkS)z(Analog of 3.7+ self.ip_network.subnet_of)rOr5rAr`�broadcast_address)r�rCrDrA�a�bs      rE�	subnet_ofzIPList.subnet_of:sQ���O���c�4��1�1��
�I���"�
;��!�Q�%6�6�
;��#�q�':�:�	
rGc�d�|�
Jd���|�
Jd���t|jpd|��|_t|jpd|��|_|j|k}||_|�||_|�|���|jj�||���|j|jd�S)zT
        Update blocking properties

        :return tuple: real expiration
        Nz'expiration' must not be Nonez'deep' must not be Noner)�force_insertr�)rqr)	�maxrqrr�r��saver�r�r�)r�rqrr�r��primary_key_changeds      rE�update_propertieszIPList.update_propertiesDs����%�%�'F�%�%�%����!:�����d�o�2��J�?�?�����	��Q��-�-��	�"�m�x�7�� ��
��� �D�K�
	
�	�	�2�	�3�3�3����!�!�(�t�!�4�4�4��/��I�
�
�	
rGc���t|��\}}}|�|j|����d�����|���|j|k|j|k|j	|k���
td��������
d��}|D]
}|jcSdS)zReturn the name of highest priority list that contains the *ip*.
        Return None if *ip* not in any list or record expired.r�r:N)r4r�r�r7rr�rrr`r@rArrr6r+)r�rQrCrDrAr��rs       rE�effective_listzIPList.effective_listas���
-�R�0�0���T�7��J�J����!�!�#�#�)�)�*�5�5�
�
��U����!�!�!��#�s�*���t�#���w�&�	���X�c�*�o�o�*�*�,�,�
-�
-�
�U�1�X�X�	
��	�	�A��:�����trGc��|jj���5|�|||��cddd��S#1swxYwYdS)z.Update ip lists on CaptchaDosAlert atomically.N)�_metar��atomic�*_blacklist_graylisted_on_captcha_dos_alert)r�rQrqr	s    rE�)blacklist_graylisted_on_captcha_dos_alertz0IPList.blacklist_graylisted_on_captcha_dos_alertys���
�Y�
�
&�
&�
(�
(�	�	��A�A��J����	�	�	�	�	�	�	�	�	�	�	�	����	�	�	�	�	�	s�A�A�
Ac���t���\}}}g}t��d���}|D�]H}	|	jtjkr;|	�t
j����rtd��d���cS|	jtjkrB|	�t
j����r|	j	rtd��d���cS|	jtjkrH|	�|��s|	�
|��st��d|	j�d|����cS|	j�kr6|	�|��s!|�
|	j|	jf����J|D] \}
}|�|
|g��}|sJ��!t j�|��|kt j|kt j|kf}
t!�t j��j|
����}|d�|D��z
}t!���j|
����|r&t2�d	�d
�|D����|��tj||d���t2�d
���t9�tjft9|���i�fd�|D�����S)a�Update ip lists on CaptchaDosAlert.

        Spec [1]:
        if search(ip, "WHITE"):  # should not really happen
            return

        existing_black_supernets = search(ip, "BLACK")
        if any(n.expiration >= expiration for n in existing_black_supernets):
            # should not really happen
            return

        existing = search_exactly(ip, "BLACK")
        if existing.manual:
            # Do nothing if already added manually
            return
        else:
            # exact match with less expiration, remove it
            # and replace with new expiration later
            remove(ip, "BLACK")

        # it can really exist only in GRAY list
        for listname in ["GRAY", "GRAY_SPLASHSCREEN", "IGNORE"]:
            existing = search_exactly(ip, listname)
            if existing and existing.expiration <= expiration:
                remove(ip, listname)

        add(ip, "BLACK", expiration)

        [1]: https://gerrit.cloudlinux.com/#/c/61260/14/src/handbook/message_processing/local_captcha_dos.py

        :param ip: attackers ip
        :param expiration: when record will expired
        :return: Union[Dict, Exception]
        r:rpz!Don't blacklist whitelisted ips [z] on CaptchaDosAlertz*Don't blacklist manually blacklisted ips [z. is already blacklisted for long enough time: z >= c�4�g|]}|jtjf��Sr[)rQrmr�)r\rks  rEr�zEIPList._blacklist_graylisted_on_captcha_dos_alert.<locals>.<listcomp>�s!��F�F�F�!���v�}�-�F�F�FrGzRemoved %s from %s listsc��g|]\}}|��Sr[r[)r\�_�Ls   rEr�zEIPList._blacklist_graylisted_on_captcha_dos_alert.<locals>.<listcomp>�s��0K�0K�0K�t�q�!��0K�0K�0KrGF)rQr�rqr	r�zPut %s on the BLACK listc���g|]	\}}�|f��
Sr[r[)r\rur�rQs   �rEr�zEIPList._blacklist_graylisted_on_captcha_dos_alert.<locals>.<listcomp>�s!���G�G�G�K�A�x�"�h��G�G�GrG)�	blocklist�unblocklist)r4rm�find_closest_ip_netsr�r�rZror(r�r�r^rqrOr$rQr��
IgnoreListr`�bin_andr@rAr�r�r�r��logger�infor��dict)r�rQrqr	�networkrDrAry�	supernetsrC�ip_r�r��	is_subnet�ignore_subnetss `             rErpz1IPList._blacklist_graylisted_on_captcha_dos_alert�s[���L"1��!4�!4����w����/�/��q�/�A�A�	��$	;�$	;�C��|�v�|�+�+��0@�0@�����0M�0M�+��v��r�r�������
����,�,��$�$�T�Y�[�[�1�1�-��J�-��v��r�r�������
�|�v�|�+�+�� � ��,�,�,�47�N�N�:�4N�4N�,�
��������"�
�	�������~��#�#�C�,<�,<�Z�,H�,H�#��"�"�C�F�C�L�#9�:�:�:��)�	 �	 �M�C���/�/��h�Z�@�@�L����<��
�&�.�.�t�4�4��?���$�&���'�)�
�	�
���j�m�,�,�2�I�>�F�F�H�H�	�	�F�F�~�F�F�F�F�������!�9�-�5�5�7�7�7��	��K�K�*�B�0K�0K�{�0K�0K�0K�
�
�
�
	�
�
���\�!���	�	
�	
�	
�	���.��3�3�3���V�\�"�D�J�$?�$?�$?��H�G�G�G�;�G�G�G�
�
�
�	
rGc�B�t|j|j|j��SrM�r5r`r@rAr�s rErOzIPList.ip_network�s"�� �� �$�,���
�
�	
rG)rqr�c�,�t|��\}}}|����t||||f����}	|	�|�|����}	|�-|	�|j�|����}	|	�|j�	����}	|�|	�|j
|k��}	|�|	�|��}	t|	��S)a�
        Returns all supernets containing given network (*ip*)
        that are not expired by *expiration* time.

        :param ip: ip network to lookup
        :param listname: list of listnames
        :param limit: number of supernets to return
        :param expiration: seconds since the epoch or None
           None means "use the current time"
        :return: list of matching supernets, ordered by netmask
                 (desc: from smallest to largest)
        )
r4r�r�rKrrr�r�rr@r6r�r+r)
r�rQr�r+rqr�rCrDrAr�s
          rErzzIPList.find_closest_ip_netss���,-�R�0�0���T�7��J�J�L�L���$�S�3��g�*>�?�?�
�
��
�G�G�S�^�^�J�/�/�/�0�0���������(�(��2�2�3�3�A�
�J�J�s�{�'�'�)�)�*�*��������
�f�,�-�-�A���������A��A�w�w�rGc�F��td�fd��jD����S)Nc�0��g|]\}}�j|k|f��Sr[r�)r\r�r�r�s   �rEr�z(IPList.list_priority.<locals>.<listcomp>*s9���
�
�
�&�H�h����)�8�4�
�
�
rG)rr�rUs`rEr7zIPList.list_priority&sA�����
�
�
�
�*-�*@�
�
�
�
�
�	
rG�networksc#�K�t�|��t�tjtjtj���tjdktjtkztjdktjtkzzt�
|��z��}|�.|�tjtjk��}n2|�tj�
|����}|�ttjtjktj�tj��tjkztjtjkztjtjktjtjkztjtjkztjtjkzz|�dn!|���tjkz���}t)|�����D]
}t-|�V��dS)z�Yield networks which has lasting supernets with higher priority in
        db.

        If listname isn't provided ignore priorities (for unblock).

        Implemented to solve performance issue: DEF-15123
        r"�NTr)�_TempIPList�fillrmr�r`r@rAr��IPV4_HOST_MASK�IPV6_HOST_MASKrrr�r�r�rr|rqrnr7r��set�tuplesr5)r�r�r�r�r�r�s      rE�filter_ips_has_supernetszIPList.filter_ips_has_supernets0s����	����"�"�"��M�M��'�����
�
��%��.�A�%�&�.�N�*J�K��N�a�'�F�N�n�,L�M�O��!�!�-�0�0�0�	
2�
�
�		
��������6�<�7�8�8�A�A������+�+�I�6�6�7�7�A�
�F�F����;�#6�6��/�7�7���G�G��-�.��
�>�K�$7�7�9�$�.�&�,�>�%�0�K�4J�J�L�"�,���<�	>�#�-���=�?��$!�(��D��+�+�-�-��1E�E�'�
�
�
��8�1�8�8�:�:���	.�	.�G�#�W�-�-�-�-�-�	.�	.rGc#�pK�t�|��|�tjtjtj���t���tj	z���
ttjtjktjtjkztjtjkzt���tjkztj
tjktj
tj
kztj
tjkzztj
tjkz���}t|�����D]
}t#|�V��dS)z�Yield exact match of ips which already recorded in db with the same
        priority and later expiration (so we don't need to add them)

        Implemented to solve performance issue: DEF-15123
        rN)r�r�r�r`r@rAr�rmrrr�rr7r�rqrnr�r�r5)r�r r�r�s    rE�find_ips_with_later_expirationz%IPList.find_ips_with_later_expirationmsa����	��������J�J��+��#��#�
�
�
�U�V�&�&�(�(�(�f�m�^�<�
=�
=�
�T���^�{�':�:��-��1L�L�N��~��)<�<�>��+�+�-�-��1E�E�G�$�.�&�,�>�%�0�K�4J�J�L�"�,���<�	>�
�#�-���=�?����	
�6�1�8�8�:�:���	.�	.�G�#�W�-�-�-�-�-�	.�	.rGc
#�K�t�|��|�tjtjtjtjtjtjktjtjktjtjkztjtjkzz���ttjtjktj�	tj��tjkztjtjkzt�
��tjkztjtj
ktjtjkztjtj
kzz���}|���D]\}}}}}t|||��||fV��dS)aFYield
            - subnet (include self) with less priority
                (listname and less expiration),
            - listname of the subnet
            - should_unblock which is True if subnet is exact blocked
                network with same listname

        Implemented to solve performance issue: DEF-15123
        rN)r�r�r�rmr`r@rAr�rr|r7r�rqrnr�r5)r�r r�r`r@rAr��should_unblocks        rE�"find_ip_subnets_with_less_priorityz)IPList.find_ip_subnets_with_less_priority�s�����	��������J�J��"��N��N��O��%���8� �(�F�N�:�"�2�f�6L�L�N��~��)<�<�>���


�

��$����;�#6�6��*�2�2�;�3F�G�G�"�2�3��
�>�[�%8�8�:��'�'�)�)�[�-A�A�
C� �*�f�l�:�!�,��0F�F�H�#�-���=�	?�
��
�
�	
�N�X�X�Z�Z�
		(�		(�
������#���'�����(�
(�
(�
(�
(�		(�		(rG)r�rqc�*�t|��\}}}|�|j���|j|k|j|kz|j|kz��}|�|�|����}|�-|�|j�|����}|�	|j�
����}|�|�|j|k��}t|��S)z�
        Returns all lists containing given network (*ip*)
        :param ip: ip network to lookup
        :param listname: list of listnames
        :return: names of list
        )
r4r�r�r�r`r@rArrr�rr6r�r)	r�rQr�r�rqrCrDrAr�s	         rE�
find_listszIPList.find_lists�s���-�R�0�0���T�7��J�J�s�|�$�$�*�*�
�
 �C�
'��{�d�"�
$��{�g�%�
'�
�
��

�G�G�S�^�^�J�/�/�/�0�0���������(�(��2�2�3�3�A�
�J�J�s�|�(�(�*�*�+�+��������
�f�,�-�-�A��A�w�w�rGc�P��t��j|it|����SrM�r�rhrj�r��queryr^r�s   �rErhz
IPList.get��'����u�w�w�{�E�J�%A�&�%I�%I�J�J�JrGc�P��t��jdit|����S�Nr[�r��
get_or_createrj�r�r^r�s  �rEr�zIPList.get_or_create��)���$�u�w�w�$�L�L�'C�F�'K�'K�L�L�LrGc�P��t��jdit|����Sr��r��
create_or_getrjr�s  �rEr�zIPList.create_or_get�r�rG)�include_itselfr��
expired_byc��	�t|��\}}�	|�|j|j|j|j���|rt|||�	f��nt|||�	f����}|�)|�|�	����}n8|tjkr(|�|�	|����}|�-|�|j�|����}|�|�|j
|k��}�	fd�|���D��S)arReturn ip_network objects containing all
        *ip* entries expired by *expired_by* from lists *listname*
        which are members of net *ip* including itself if *include_itself*

        :param ip: network to lookup members for
        :param listname: list name
        :param include_itself: whether to include ip itself as a subnet
          [default: False]
        :param expired_by: expiry date as "seconds since epoch"
           return entries expired by given *expired_by* timestamp
           - IPList.NEVER :: return all entries (regardless expiration)
           - None :: return non-expired entries

        Nc�B��g|]\}}}}t||���||f��Sr[)r5)r\rCrDr��erAs     �rEr�z+IPList.find_net_members.<locals>.<listcomp>$sD���
�
�
�&��T�8�Q��s�D�'�
2�
2�H�a�@�
�
�
rG)r4r�r`r@r�rqr�rFrIrrrmrnr�r�r�)
r�rQr�r�r�r�rCrDr�rAs
         @rE�find_net_memberszIPList.find_net_members�sN���0-�R�0�0���T�7��J�J�����c�l�C�N�
�
�
�%��
M�"�3��d�G�(<�=�=�=�1�#��T�7�7K�L�L�
�
�	
���������)�)�)�*�*�A�A�
�6�<�
'�
'�������z�2�2�3�3�A��������(�(��2�2�3�3�A�������
�f�,�-�-�A�
�
�
�
�*+�(�(�*�*�
�
�
�	
rGc�T�|j�|��}|j|d�S)zQReturn list of iplists with less priority than list from given
        propertiesN��IP_LISTS�index�r�r�r�s   rE�#lists_with_less_or_equal_prioritiesz*IPList.lists_with_less_or_equal_priorities)s*����"�"�8�,�,���|�E�F�F�#�#rGc�Z�|j�|��}|jd|dz�S)zRReturn list of iplists with greater priority than list from given
        listnameNr:r�r�s   rE�&lists_with_greater_or_equal_prioritiesz-IPList.lists_with_greater_or_equal_priorities2s.����"�"�8�,�,���|�K�e�a�i�K�(�(rGc�,�t|��\}}}|����|j|k|j|k|j|k|j|k��}|�|�|j|k��}|���SrM)	r4r�r�r�r`r@rAr�r�)r�rQr�r�r�rDrAr�s        rE�removez
IPList.remove<s���!0��!4�!4����w��
�
���"�"��L�H�$���7�*��K�4���K�7�"�	
�
�����K�K��
�f� 4�5�5�E��}�}���rG�to_blockc���tdt|��|j��D]L}|�||||jz����d������MdS)Nr�REPLACE)�range�len�
BATCH_SIZE�insert_many�on_conflictr�)r�r��idxs   rE�
block_manyzIPList.block_manyIsq����C��M�M�3�>�:�:�	�	�C��O�O�H�S�3���+?�%?�@�A�A�M�M��
�
��g�i�i�i�i�		�	rGc�t��t�|��td�d��jjj������}�fd��jjjD��}�j|��	tj���ttjtjktj
tj
kztjtjkztjtjkz���}g}|���D]-\}}}}	|�t%|||��|	f���.�����	|�|�������|S)zj
        Remove *ips* that are not manual from the [iplist] table.
        Return ips to unblock.
        z({})z, c3�8�K�|]}t�|��V��dSrM)rD)r\rEr�s  �rEr_z%IPList.remove_many.<locals>.<genexpr>]s>�����
�
�$)�G�C����
�
�
�
�
�
rGr)r�r�r�formatrrnr��field_namesr�r�rmr�rAr`r@r�r�r$r5r�r�r�)
r�r �primary_key_sqlr��	to_remove�
to_unblockr`r@rAr�s
`         rE�remove_manyzIPList.remove_manyQs����	���������M�M�$�)�)�C�I�$9�$E�F�F�G�G�
�
��
�
�
�
�-0�Y�-B�-N�
�
�
��
�C�J��$�
�U�F�M�>�
"�
"�
�T���^�{�':�:��-��1L�L�N��~��)<�<�>���+�*>�>�@����	��
�;D�;K�;K�;M�;M�	�	�7�O�W�g�x����%�o�w��H�H���
�
�
�
�	�
�
�����?�.�.�y�9�9�:�:�B�B�D�D�D��rG)rFrM�NN)NNNNNN)uryrzr{r|r�r�r�r�r�r�r�	enumerate�reversedr�r
r#r��IPv4�VERSION_IP4�IPv6�VERSION_IP6r�r�rrQrr�rr�rnr!rqrrrr	r
rr�r�r�rr`r@rArr�r�r�rr�r�r�rr�r�r�r�rr�r3r�boolr�r�rrrr�r�r�r�rrr)�	frozensetrr&rBrFrIrNr
rPrRrVrrrZr\r^rcrirlrqrp�propertyrOrNrzr7rrr�r�r�r�rhr�r�r�rr�r�r�r�r��
__classcell__�r�s@rErmrm�s��������;�;� �K�
�E�
�E��D�+���u�d�$5�6�H���i�i����(:�(:�;�;�<�<���K��K�
�F��K��K��J���������&
���	�	�	�B��y�
��U�/�6�6�u�z�z�(�7K�7K�L�L�M�M�N����H�
�E��������J�
�I�4�(�(�(�M��L�
�3�3�
�
�
�E�
�<�T�"�"�"�D��i�T�"�"�"�G��i�T�|�<�<�<�G�"�\�u�e�<�<�<�N��\�u�d�
3�
3�
3�F��,�D�)�)�)�K�#�|��u�=�=�=��#�l��.�.�.�O��l��&�&�&�G��l��&�&�&�G�
�I�
��E�E�K�K�K����E�F�F�
�
�
�
�E���������L�L�L�L�L�L�L�L�
�(�(�3�-�(�C�(�(�(��\�(���s��s�����\���	
�h�t�n�	
�	
�	
��\�	
��	
����	
�3�	
�	
�	
��\�	
��!�E�#�s�(�O�!�!�!��[�!���!�
;�;��;��#�Y�;��	;�
�;��
;�;�;��[�;�z������[���
�	���#�{�K�/�0���s�)���	����[��:����[������[��B�������
�����[��&�������?�?�?��[�?�B�B�D��I�B�B�B��[�B���t�I������[��,�!%�#�&*�%.�Y�[�[�
5�5���9�5���
�5���}�	5�
�7�#�5��E�
�
5�5�5��[�5�n�����[�������[�������[����D���	�)�	�����[�� ����[���A�A��[�A��
�
�
��[�
�
�
�
��

�

��[�

�
�
�
�
�
�
�
�
�
�
�:��	��h�s�m�����[��.����[���u
�u
��[�u
�n�
�
��X�
�
�#��	"���"�"�"��)�'��)>�>�?�"��s�)�"��	"�"�"��[�"�H�
�
��[�
��%)��	:.�:.��y�$�s�C�x�.�0�1�:.��D�>�:.�:.�:.��[�:.�x�&.��y�$�s�C�x�.�0�1�&.�&.�&.��[�&.�P�8(��y�$�s�C�x�.�0�1�8(�8(�8(��[�8(�t���
������s�)�����[��:�K�K�K�K��[�K��M�M�M�M��[�M��M�M�M�M��[�M��#�+
�
���+
�+
�+
��+
��s�)�+
��
+
�
�e�I�s�C�'�(�	)�+
�+
�+
��[�+
�Z�$��$�	�#��$�$�$��[�$��)��)�	�#��)�)�)��[�)��
�
�	�
�S�
�
�
��[�
���$�t�*�����[���$��5��D��1�2�$�	
�e�I�s�N�#�	$�$�$�$��[�$�$�$�$�$rGrmc��eZdZed���Zed���Zed���Zed���Zed���Z	edd���Z
Gd�d��Zed���Z
ed	���Zed
eeeffd���ZdS)
r�Fr�rTr�c�<�eZdZejZdZedddd��ZdS)�_TempIPList.Meta�
tmp_iplistr`r@rAr�N)	ryrzr{r*r�r�r�rr�r[rGrEr�r��s4�������;����"�l��y�)�Z�
�
���rGr�c�D�|jj�d��dS)Na�
            CREATE TEMPORARY TABLE IF NOT EXISTS tmp_iplist
            (
                network_address INT, netmask INT, version INT,
                listname TXT VARCHAR(255) NOT NULL CHECK
                (listname in ('WHITE','BLACK','GRAY','GRAY_SPLASHSCREEN')),
                priority INT,
                expiration INT,
            PRIMARY KEY (network_address, netmask, version, listname))
        )rnr��execute_sqlrUs rE�_createz_TempIPList._create�s.���	��&�&�	
�	
�	
�	
�	
�	
rGc�R�|������dSrM)r�r�rUs rE�_clearz_TempIPList._clear�s"���
�
���������rGr c�$�t|d��r|���n|}tdt��5|���|���d�|D��}ddd��n#1swxYwY|r�tdt��5d}t
dt|��|��D]4}|�||||z����	���5	ddd��dS#1swxYwYdSdS)N�itemszprepare tmp_iplist to fillc���g|]g\}}ttgd�gt|���t�|���t�|���������hS))r`r@rAr�r�rq)r�zipr4rmr�r�)r\rQ�props   rEr�z$_TempIPList.fill.<locals>.<listcomp>�s������$�B��#������,�R�0�0��#�C�C�D�I�I��#�6�6�t�<�<��������rGzfill tmp_iplist�r)
�hasattrr�r.r}r�r�r�r�r�r�)r�r �data�
batch_sizer�s     rEr�z_TempIPList.fill�s���$�S�'�2�2�;�c�i�i�k�k�k���
�0�&�
9�
9�	�	��K�K�M�M�M��J�J�L�L�L���$!$�%���D�	�	�	�	�	�	�	�	�	�	�	����	�	�	�	�0�	L��)�6�2�2�
L�
L� �
� ��C��I�I�z�:�:�L�L�C��O�O�D��s�Z�/?�)?�$@�A�A�I�I�K�K�K�K�L�
L�
L�
L�
L�
L�
L�
L�
L�
L�
L�
L�
L����
L�
L�
L�
L�
L�
L�	L�	Ls$�5A=�=B�B�AD�D�
DN)ryrzr{r!r`r@rArr�r�rqr�r�r�r�rrr
r�r[rGrEr�r�ys������"�l��.�.�.�O��l��&�&�&�G��l��&�&�&�G��y�e�$�$�$�H��|��'�'�'�H��������J�
�
�
�
�
�
�
�
��
�
��[�
�����[���L�u�T�8�^�,�L�L�L��[�L�L�LrGr�c��eZdZdZd\ZZedd���Zedd���Z	Gd�d	��Z
ed
���Zed���Z
dS)
�LastSynclistzFUsed to track how up-to-date are lists synced from Correlation server.)rQ�hashTrr�F)r�r�c�$�eZdZejZdZdZdS)�LastSynclist.Meta�
last_synclistr�N)ryrzr{r*r�r�r�r�r[rGrEr�r��s�������;��"�����rGr�c��|�tj������|j|k�����dS)N)�	timestamp)rdror��namer�)r�r�s  rE�update_timestampzLastSynclist.update_timestamp�sC���
�
�T�Y�[�[�
�)�)�/�/���D�0@�A�A�I�I�K�K�K�K�KrGc�H�|�|ddi���\}}|jS)Nr�r)r��defaults)r�r�)r�r��objrus    rE�
get_timestampzLastSynclist.get_timestamp�s+���"�"���Q�7G�"�H�H���Q��}�rGN)ryrzr{r|r7�HASHrr�rr�r�r�r�r�r[rGrEr�r��s�������P�P��H�B���
��a�0�0�0�I��9�%�T�2�2�2�D���������
�L�L��[�L�����[���rGr�c��eZdZdZGd�d��Ze��Zed���Ze	d���Z
e	d���ZdS)	�WhitelistedCrawlerz^
    Crawlers for which local alerts must not add IP
    to the :attr:`IPList.GRAY` list.
    c� �eZdZejZdZdS)�WhitelistedCrawler.Meta�whitelisted_crawlersN�ryrzr{r*r�r�r�r[rGrEr�r��s�������;��)���rGr�Fr�c��tj���5|�|������}|D]}t
�||����	ddd��dS#1swxYwYdS)N)�description)�
crawler_id�domain)r*r�ro�insertr��WhitelistedCrawlerDomainr�)r�r��domains�inserted_id�ds     rE�addzWhitelistedCrawler.add�s���
�[�
�
�
!�
!�	�	��*�*��*�=�=�E�E�G�G�K��
�
��(�/�/�*�1�0�����
�	�	�	�	�	�	�	�	�	�	�	�	����	�	�	�	�	�	s�A
A7�7A;�>A;c��|����|j���|���|��}t
���}t
||��}g}|�d���}|D]6}|j|jd�|j	D��d�}	|�
|	���7||fS)NT)�clear_limitc��g|]	}|j��
Sr[)r)r\rs  rEr�z,WhitelistedCrawler.fetch.<locals>.<listcomp>�s��>�>�>��A�H�>�>�>rG)rr�r)r�rr�r+r*rr%rrrr$)
r�r+r*�crawlers_query�
domains_query�crawlers_with_domains_queryr%�	max_count�crawler�items
          rErBzWhitelistedCrawler.fetch�s���
�J�J�L�L�!�!�#�/�2�2�8�8��?�?�F�F�v�N�N�	�1�7�7�9�9�
�&.�~�}�&M�&M�#���"�(�(�T�(�:�:�	�2�	 �	 �G��j�&�2�>�>�g�o�>�>�>���D�

�M�M�$������&� � rGN)ryrzr{r|r�r"rr#r�r�rrBr[rGrEr�r��s���������
*�*�*�*�*�*�*�*�
��	�	�B��)��'�'�'�K�����[���!�!��[�!�!�!rGr�c�v�eZdZdZGd�d��Ze��Zeeddd���Z	e
d���Zd	S)
rzBDomain names used to check if IP is a :class:`WhitelistedCrawler`.c� �eZdZejZdZdS)�WhitelistedCrawlerDomain.Meta�whitelisted_crawler_domainsNr�r[rGrEr�rs�������;��0���rGr�F�CASCADEr�r��	on_delete�related_namer�N)ryrzr{r|r�r"rr r�rr#rr[rGrErrs�������L�L�1�1�1�1�1�1�1�1�
��	�	�B��o��
���	���G��Y�E�
"�
"�
"�F�F�FrGrc	���eZdZdZdZdZed���Zeded�	ee����g���Z
edd�	��ZGd
�d��Z
eded
ededefd���ZdS)�RemoteProxyGroupz9Groups multiple remote proxies together with common data.r��
imunify360Fr�zsource in ('{}', '{}')r�Tr�c�$�eZdZejZdZdZdS)�RemoteProxyGroup.Meta�remote_proxy_group))�r��sourceTN)ryrzr{r*r�r�r��indexesr[rGrEr�r)s�������;��'��/���rGr�r�r �enabledrkc��t�||���}|j|krdS||_|���dS)z�Set group's enabled status.

        Group is identified by name and source. Returns True if enabled
        status has changed, False otherwise (it was a noop).rFT)rrhr"rg)r�r�r r"r�s     rE�set_enabledzRemoteProxyGroup.set_enabled.sG��!�$�$�$�v�$�>�>���=�G�#�#��5���
�
�
�
�����trGN)ryrzr{r|�MANUAL�
IMUNIFY360rr�rr�r rr"r�r�r�r�r$r[rGrErrs�������C�C��F��J��9�%� � � �D��Y�
��E�*�1�1�&�*�E�E�F�F�
����F��l��t�4�4�4�G�0�0�0�0�0�0�0�0�
�
�s�
�C�
�$�
�4�
�
�
��[�
�
�
rGrc
��eZdZdZeed���Zed���ZGd�d��Z	e
deedeedee
d	eefd
���Ze
deded
eefd���Ze
ded
eefd���ZdS)�RemoteProxyz!Remote Proxy networks in a group.Fr�c� �eZdZejZdZdS)�RemoteProxy.Meta�remote_proxyNr�r[rGrEr�r*Cs�������;��!���rGr��by_group�	by_sourcer"rkc��|�tjtjtj|j���t��}|�#|�tj|k��}|�#|�tj|k��}|�#|�tj|k��}t|�	tj���
����S)z�Returns a list of remote proxy networks as dicts.

        Results are optionally filtered by group name, source, and enabled
        status.)r�rr r�r"r�rr�rrrK)r�r,r-r"r�s     rErzRemoteProxy.listGs���
�J�J��#��!��$��K�	
�
�
�$��
 �
 �	
������(�-��9�:�:�A�� ����(�/�9�<�=�=�A������(�0�G�;�<�<�A��A�J�J�/�4�5�5�;�;�=�=�>�>�>rGr�r r�c���t�||���\}}|D]R}tjt	j|����}t
||j���}|����SdS)z>Adds networks to a list of remote proxy in group name, source.r)r��group_idN)	rr�r7rbrNrOr(rrg)r�r�r r�r�rurC�proxys        rE�add_manyzRemoteProxy.add_many`sw��$�1�1�t�F�1�K�K���q��	�	�C��%�i�&:�3�&?�&?�@�@�C���e�h�?�?�?�E��J�J�L�L�L�L�		�	rGc���d�|D��}g}t����t���tj|z���tj|k��}t|��D]0}|�|j��|�	���1t�t���ttj���t���
tjtj��dk��}t|��D]}|�	���|S)zrDeletes networks from remote proxy lists.

        Only networks coming from groups with given source are deleted.c�Z�g|](}tjtj|������)Sr[)r7rbrNrO)r\rCs  rEr�z/RemoteProxy.delete_networks.<locals>.<listcomp>ps;��
�
�
�?B�B��	� 4�S� 9� 9�:�:�
�
�
rGr)r(r�rrr�r�r rr$�delete_instancerrrrr$�COUNTr)r�r r�r�r�r1r�s       rE�delete_networkszRemoteProxy.delete_networksjs=��
�
�FN�
�
�
����
��� � �
�T�"�
#�
#�
�U�;�&�(�2�
3�
3�
�U�#�*�f�4�
5�
5�		
��!�W�W�	$�	$�E��N�N�5�=�)�)�)��!�!�#�#�#�#�
�#�#�$4�5�5�
�T�+�t��
/�
/�
�X�&�
'�
'�
�V�B�H�[�^�,�,��1�
2�
2�		
��!�W�W�	$�	$�E��!�!�#�#�#�#��rGN)ryrzr{r|r rr�r#r�r�r�rr�r�rrrr2r7r[rGrEr(r(<s*������+�+��O�,�5�9�9�9�E��i�U�#�#�#�G�"�"�"�"�"�"�"�"��?��3�-�?��C�=�?��$��	?�

�d��?�?�?��[�?�0��C�����S�	�����[����S��D��I�����[���rGr(c�N��eZdZdZed���Zed���Zed���Zed���Z	Gd�d��Z
e�fd���Ze�fd���Z
e�fd���Ze�fd	���Zed
edeefd���Zed
eefd���Z�xZS)r{z�
    IP addresses from this list are not blocked by firewall. However,
    they still can be placed to other lists by either server or local
    events or by user request.
    Fr�c�>�eZdZejZdZeddd��ZdZ	dS)�IgnoreList.Meta�ignore_listr`r@rAr�Nr�r[rGrEr�r:�s3�������;�� ��"�l�#4�i��K�K�����rGr�c�P��t��jdit|����S)r�r[�r�r�rer�s  �rEr�zIgnoreList.create�s*����u�w�w�~�9�9� 0�� 8� 8�9�9�9rGc�P��t��jdit|����Sr�r�r�s  �rEr�zIgnoreList.create_or_get�r�rGc�P��t��j|it|����SrMr�r�s   �rErhzIgnoreList.get�r�rGc�P��t��jdit|����Sr�r�r�s  �rEr�zIgnoreList.get_or_create�r�rG�supernetrkc#�,K�t|��\}}}|����|j�|��|k|j|k|j|k��}|D]$}t|j|j|j��V��%dSrM)r4r�r�r`r|r@rAr5)r�rA�addressrDrAr�r@s       rE�subnetszIgnoreList.subnets�s�����!0��!:�!:����w��J�J�L�L���
�
 �
(�
(��
.�
.�7�:��K�4���K�7�"�
�
��
�	�	�C�#��#�S�[�#�+���
�
�
�
�	�	rG�	to_deletec��t|��}|D]k}t|��\}}}|����|j|k|j|k|j|k������ldSrM)r�r4r�r�r`r@rAr�)r�rE�uniquerrCrDrAs       rEr�zIgnoreList.remove�s����Y�����	�	�D�%4�T�%:�%:�"�G�T�7��J�J�L�L����#�w�.���t�#���w�&�
�
��g�i�i�i�i�
	�	rG)ryrzr{r|rrQr!r`r@rAr�r�r�r�rhr�r3r
rDrr�r�r�s@rEr{r{�s����������
���	�	�	�B�"�l��.�.�.�O��l��&�&�&�G��l��&�&�&�G����������:�:�:�:��[�:��M�M�M�M��[�M��K�K�K�K��[�K��M�M�M�M��[�M��
�y�
�X�i�-@�
�
�
��[�
���t�I������[�����rGr{c���eZdZdZed���Zedede�de	�de
�d���g���Zed���ZGd	�d
��Z
e	dd���Zed
���Zedd���ZdS)�BlockedPortz]Port blocking configuration.

    Effective when `FIREWALL.port_blocking_mode == ALLOW`.
    Fr�zproto in ('z', 'r�r�Tc�(�eZdZejZdZdZdZdS)�BlockedPort.Meta�blocked_portr�)))�port�protoTN�	ryrzr{r*r�r�r�r�r!r[rGrEr�rK�s&�������;��!����
���rGr�Nc�B�|�M|�|j�|��tj�|��z��}|�(|�t	t|����}|�#|�t
j|k��}|SrM)r�r	rR�
IgnoredByPortrVr/r)r�r�rrrs     rE�_add_filter_argszBlockedPort._add_filter_args�s����!������$�$�Z�0�0��'�0�0��<�<�=���A������,�]�E�B�B�C�C�A��&�������7�8�8�A��rGc�^�|�|j������tt
j���tt
jtjtjk���}|j	|fi|��}|�
��S)Nr)r�r�distinctrrQrrr/r
rRr)r�rr�s   rErzBlockedPort.fetch_count�s���
�J�J�s�v���
�X�Z�Z�
�T�-���
1�
1�
�T����!�)�W�Z�7����		
�
!�C� ��2�2�k�2�2���w�w�y�y�rGr�rc
��|�|j|j|j|jt
jt
j�d�����t
tj
���ttj
t
jtjk���}|j
|fi|��}|�|j|jt
j��}d�}g}tj|���|���D]+\}}d�|D��}	|	|d<|�|���,||||z�S)N�
ip_commentrc� ���fd�dD��S)Nc�"��i|]}|�|��Sr[r[)r\�keyr@s  �rE�
<dictcomp>z8BlockedPort.fetch.<locals>.group_key.<locals>.<dictcomp>s,������"%��S��X���rG)rrMrNr	r[�r@s`rE�	group_keyz$BlockedPort.fetch.<locals>.group_keys.�������)K����
rG)rYc�@�g|]}|d�
|d|dd���S)rQNrV)rQr	r[)r\rQs  rEr�z%BlockedPort.fetch.<locals>.<listcomp>$s=�������d�8�'��$�x�B�|�,<�=�=�'�'�'rGr )r�rrMrNr	rQrQrrrrr/r
rRr�	itertools�groupbyrKr$)
r�r+r*rr�r\r%�
port_protor �ignored_ipss
          rErBzBlockedPort.fetchsP��
�J�J������	���� ��%�+�+�L�9�9�

�
��T�-���
1�
1�
�T����!�)�W�Z�7����	
�"
!�C� ��2�2�k�2�2��
�J�J�s�x���M�,<�=�=��	�	�	�
��(�0������	�J�J�J�	&�	&�O�J��������K�
!,�J�u���M�M�*�%�%�%�%��f�v��~�-�.�.rG)NNN)r�r)ryrzr{r|r!rMrrr1r2r0rNr	r�r�rRrrBr[rGrErIrI�s���������<�U�#�#�#�D��I�
��U�B��B�B�#�B�B�3�B�B�B�C�C�D�
�
�
�E�
�i�T�"�"�"�G�
�
�
�
�
�
�
�
��=A�����[������[���%/�%/�%/��[�%/�%/�%/rGrIc�2��eZdZdZeeddd���Zed���Zed���Z	e
d���Ze
d���Ze
d���Z
edd�	��ZGd
�d��Ze�fd���Zeddefd���Zedd���Z�xZS)rQz7Ignored IPs for ports blocked via :class:`BlockedPort`.Frr rr�Tr�r�c�(�eZdZejZdZdZdZdS)�IgnoredByPort.Meta�ignored_by_port_protor�)))r`rQTNrOr[rGrEr�rdCs&�������;��*����
���rGr�c�T��t��jdit|����dSr�r=r�s  �rEr�zIgnoredByPort.createMs.��������2�2�)�&�1�1�2�2�2�2�2rGNrAc��|�|t���t��}|�|�|j|k��}|SrM)r�rIrr�rA)r�rAr�s   rErBzIgnoredByPort.fetchQsF���J�J�s�K�(�(�-�-�k�:�:���������w�.�/�/�A��rGc���|jj�d�t|jj��z}|jj�d|�d���|j�|���dS)Nr4z&
          CREATE TABLE IF NOT EXISTS a�ignored_by_port_proto
          (
             "id" INTEGER NOT NULL PRIMARY KEY,
             "port_proto_id" INTEGER NOT NULL,
             "ip" VARCHAR(255) NOT NULL,
             "comment" VARCHAR(255),
             "network_address" INTEGER NOT NULL,
             "netmask" INTEGER NOT NULL,
             "version" INTEGER NOT NULL,
             "country_id" VARCHAR(255),
             FOREIGN KEY ("port_proto_id")
             REFERENCES "blocked_port" ("id") ON DELETE CASCADE
        ))�safe)rnr�r�r�r��_schema�create_indexes)r�ri�optionsr�s    rE�create_tablezIgnoredByPort.create_table]s~���I�$�'�'�'�$�s�y�/?�*@�*@�@���	��&�&�


�&,�


�


�


�	
�	
�	
� 	��"�"��"�-�-�-�-�-rGrM)T)ryrzr{r|r rIr`rrQr	r!r`r@rAr
r�r�r�r9rBrmr�r�s@rErQrQ/sN�������A�A� ���%�9�5����J�

���	�	�	�B��i�T�"�"�"�G�"�l��.�.�.�O��l��&�&�&�G��l��&�&�&�G��i�T�|�<�<�<�G�
�
�
�
�
�
�
�
��3�3�3�3��[�3����,�����[���.�.�.��[�.�.�.�.�.rGrQc	�J��eZdZdZed���Zed���Zed���Zed���ZGd�d��Z	e
	ddeded	e
efd
���Ze
deded	efd���Ze
d���Ze
dd���Ze
dd���Ze
�fd���Z�xZS)�IPListRecordz/DB table that stores ips for given iplist_id`s.Fr�c�@�eZdZejZdZedddd��ZdZ	dS)�IPListRecord.Meta�iplistrecordr`r@rA�	iplist_id�
ipsetlistsNr�r[rGrEr�rq|s;�������;��!��"�l��y�)�[�
�
�����rGr��P��
ip_versionrsrkc#��K�|�|j|j|j���|j|k|jt|kz�����}d}|�|���	|��x}r^||�
��z
}tt|��Ed{V��|�|���	|��x}�\dSdS)z�
        Yield ips corresponding to *ip_version*, *iplist_id*.
        NOTE: It assumes exclusive access to the db until the iterator
              is exhausted.
        rN)
r�r`r@rAr�rsr9r�r+r*rrr5)r�rvrs�
chunk_sizer�r*r&s       rE�	fetch_ipszIPListRecord.fetch_ips�s����(
�I�I�b�(�"�*�b�j�A�A�
�U����*��:�!1�*�!=�=�?����V�X�X�
	����{�{�:�.�.�5�5�f�=�=�=�e�	9��e�k�k�m�m�#�F��0�%�8�8�8�8�8�8�8�8�8��{�{�:�.�.�5�5�f�=�=�=�e�	9�	9�	9�	9�	9rGc��|����|j|k|jt|kz�����S)z*How many *ip_version* ips for *iplist_id*.)r�r�rsrAr9r)r�rvrss   rE�fetch_ips_countzIPListRecord.fetch_ips_count�sM��
�I�I�K�K�
�U����*��:�!1�*�!=�=�?����U�W�W�
	
rGc�p�|�|tjtj�d�����t|jtjk���}|�#|�tj|k��}|r#|�t||����}|S)Nrr)r��
IPListPurposer�rsrrr�rX�r�r�rr�s    rErzIPListRecord._fetch_query�s����J�J���!��#�)�)�$�/�/�
�
��$�}�#�-�=�3J�"J�$�
L�
L�		
������
�-��8�9�9�A��	;����-�c�5�9�9�:�:�A��rGNc�V�|�||��}|���SrMrr~s    rErzIPListRecord.fetch_count�s%�����W�e�,�,���w�w�y�y�rGc��|�||��}|�|�|��}|�|�|��}d�td�|��D��S)Nc��g|]J\}}t|j��t|j��|j|jj|jjd���KS))r`r@rArsr�)r�r`r@rA�
iplistpurposerr�)r\r@rQs   rEr�z&IPListRecord.fetch.<locals>.<listcomp>�se��
�
�
���R�
$'�r�'9�#:�#:��r�z�?�?��;� �.�1��,�4�
�
�
�
�
rGc�F�|t|j|j|j��fSrMr�r[s rEr�z$IPListRecord.fetch.<locals>.<lambda>�s'���%��+�S�[�#�+����rG)rr*r+�map)r�r�rr*r+r�s      rErBzIPListRecord.fetch�s�����W�e�,�,�������� � �A���������A�
�
�������
�
�
�	
rGc�T��t��jdit|����}|Sr�r=r�s   �rEr�zIPListRecord.create�s,����u�w�w�~�9�9� 0�� 8� 8�9�9���rG)rurMr�)ryrzr{r|r!r`r@rArsr�r�r8�IPListIDrr3ryr�r{rrrBr�r�r�s@rErorots��������9�9�"�l��.�.�.�O��l��&�&�&�G��l��&�&�&�G���%�(�(�(�I����������CH� 9� 9�!� 9�.6� 9�	�)�	� 9� 9� 9��[� 9�D�	
�	�	
�h�	
�3�	
�	
�	
��[�	
�����[�������[���
�
�
��[�
�2������[�����rGroc���eZdZdZed���Zed���ZGd�d��Ze	de
deedee
fd	���Ze	de
d
edefd���Zededefd
���ZdS)r}z6DB table that stores "purposes" for given iplist_id`s.Fr�c�<�eZdZejZdZedd��ZdZ	dS)�IPListPurpose.Metar�r�rsrtNr�r[rGrEr�r��s0�������;��"��"�l�9�k�:�:�����rGr�rv�purposesrkc���ttd��|�|j������t|jtjk����tjt|k|j
�ttt|������z�������S)z=Yield all distinct iplist_id for *ip_version* and *purposes*.rr)r�r
r�rsrTrror�rAr9r�r�rr�r�)r�rvr�s   rE�fetch_iplist_idszIPListPurpose.fetch_iplist_ids�s���
��q�M�M��I�I�b�l�#�#�
�X�Z�Z�
�T�,�B�L�L�4J�$J�T�
L�
L�
�U��%�)9�*�)E�E��*�.�.��c�#�x�&8�&8�!9�!9�:�:�;����V�X�X�

�

�
	
rGr�c�H�|�|j������t|jtjk����tjt|k|j|kz���	��S)z9How many iplists are there for *ip_version* and *purpose*r)
r�rsrTrror�rAr9r�r)r�rvr�s   rErzIPListPurpose.fetch_countsx��
�I�I�b�l�#�#�
�X�Z�Z�
�T�,�B�L�L�4J�$J�T�
L�
L�
�U��%�)9�*�)E�E��:��(�*����U�W�W�		
rGr�c���tjtjtjtjtjtjtjtji|S)z,Return purpose corresponding to iplist name.)	rmr�r�r�r}r�r~r�rr�s rE�listname2purposezIPListPurpose.listname2purposes>��
�L�'�-��L�'�,��K����$�g�&:�	
�
��	rGN)ryrzr{r|rr�r!rsr�r�r8r
r�r�r�r�rr�r�r�r[rGrEr}r}�s������@�@��i�U�#�#�#�G���%�(�(�(�I����������
�!�
�-5�g�->�
�	�(�	�
�
�
��[�
� �
�I�
��
�C�
�
�
��[�
���3��7�����\���rGr})or|rNr^�loggingro�datetimer�enumr�	functoolsrrrr�operatorr	r
�typingrrr
rrrrrrrr�blinkerr�peeweerrrrrrrrrr r!r"r#r$r%r&�playhouse.shortcutsr'�"defence360agent.contracts.messagesr(�defence360agent.modelr)r*�$defence360agent.model.simplificationr+�defence360agent.utilsr,r-r.�im360.model.countryr/�im360.utils.netr0r1r2r3r4r5r6�im360.utils.validater7r8r9r�r��	getLoggerryr}r�r��V4r�r��V6r�rFrIrKrVrXrerjr�rrrtr�r�rmr�r�r�rrr(r{rIrQror}r[rGrE�<module>r�s���2�2�����������������������������������.�.�.�.�.�.�.�.�������$�$�$�$�$�$�$�$���������������������������������������������������������������������$.�-�-�-�-�-�5�5�5�5�5�5�1�1�1�1�1�1�1�1�=�=�=�=�=�=�O�O�O�O�O�O�O�O�O�O�'�'�'�'�'�'�������������������A�@�@�@�@�@�@�@�@�@���	��	�8�	$�	$�� ����\�!:�!:�;�;�A�>�� ����W�!5�!5�6�6�q�9������$������$�����;��',�S�#�s�]�';�����$���;��',�S�#�s�]�';�����$���;��',�S�#�s�]�';�����$���&F�F�F����"���D�d�D�D�D�D�

�
�
�
�
��
�
�
������c�4����@N�N�N�N�N�U�N�N�N�b$DL�DL�DL�DL�DL�%�DL�DL�DL�N�����5����0'!�'!�'!�'!�'!��'!�'!�'!�T#�#�#�#�#�u�#�#�#�&$�$�$�$�$�u�$�$�$�NK�K�K�K�K�%�K�K�K�\A�A�A�A�A��A�A�A�H^/�^/�^/�^/�^/�%�^/�^/�^/�BB.�B.�B.�B.�B.�E�B.�B.�B.�Jp�p�p�p�p�5�p�p�p�f4�4�4�4�4�E�4�4�4�4�4rG