目录
- 1.什么是ip2region?
- Ip2region 特性
- 1、标准化的数据格式
- 2、数据去重和压缩
- 3、极速查询响应
- 4、IP 数据管理框架
- 2.代码工程
- 实验目标
- pom.XML
- controller
- dto
- Util
- 下载离线IP定位库
- 代码仓库
- 3.测试
1.什么是ip2region?
ip2region v2.0 - 是一个离线IP地址定位库和IP定位数据管理框架,10微秒级别的查询效率,提供了众多主流编程语言的 xdb
数据生成和查询客户端实现。
Ip2region 特性
1、标准化的数据格式
每个 ip 数据段的 region 信息都固定了格式:国家|区域|省份|城市|ISP
,只有中国的数据绝大部分精确到了城市,其他国家部分数据只能定位到国家,后前的选项全部是0。
2、数据去重和压缩
xdb
格式生成程序会自动去重和压缩部分数据,默认的全部 IP 数据,生成的 ip2region.xdb 数据库是 11MiB,随着数据的详细度增加数据库的大小也慢慢增大。
3、极速查询响应
即使是完全基于 xdb
文件的查询,单次查询响应时间在十微秒级别,可通过如下两种方式开启内存加速查询:
vIndex
索引缓存 :使用固定的512KiB
的内存空间缓存 vector index 数据,减少一次 IO 磁盘操作,保持平均查询效率稳定在10-20微秒之间。xdb
整个文件缓存:将整个xdb
文件全部加载到内存,内存占用等同于xdb
文件大小,无磁盘 IO 操作,保持微秒级别的查询效率。
4、IP 数据管理框架
v2.0 格式的 xdb
支持亿级别的 IP 数据段行数,region 信息也可以完全自定义,例如:你可以在 region 中追加特定业务需求的数据,例如:GPS信息/国际统一地域信息编码/邮编等。也就是你完全可以使用 ip2region 来管理你自己的 IP 定位数据。
2.代码工程
实验目标
根据来源ip 判断属于那个国家,然后做过滤
pom.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>springboot-demo</artifactId> <groupId>com.et</groupId> <version>1.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>ipfilter</artifactId> <properties> <maven.compiler.source>8</maven.compiler.source> <maven.compiler.target>8</maven.compiler.target> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-autoconfigure</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <!-- ip lib--> <dependency> <groupId>org.lionsoul</groupId> <artifactId>ip2region</artifactId> <version>2.6.4</version> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </dependency> <dependency> <groupId>com.alibaba.fastjson2</groupId> <artifactId>fastjson2</artifactId> <version>2.0.40</version> </dependency> </dependencies> </project>
controller
package com.et.ipfilter.controller; import com.et.ipfilter.util.AddressUtils; import com.et.ipfilter.util.IPOfflineUtil; import com.et.ipfilter.util.IPOnlineUtil; import lombok.extern.slf4j.Slf4j; import org.springframework.util.StringUtils; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; import Javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.ujstil.HashMap; import java.util.Map; @RestController @Slf4j public class HelloWorldController { @RephpquestMapping("/hello") public Map<String, Object> showHelloWorld(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse,String ip){ //fisrt get ip addr by offline file //String ip = IPOfflineUtil.getIpAddr(httpServletRequest); //analyze address String addr = IPOfflineUtil.getAddr(ip); if(StringUtils.isEmpty(addr)) { //get addr by online service ip = IPOfflineUtil.getIpAddr(httpServletRequest); addr= AddressUtils.getRealAddressByIP(ip); log.info("IP >> {},Address >> {}", ip, addr); // you can filter by country or province } Map<String, Object> map = new HashMap<>(); map.put("msg", "HelloWorld"); map.put("ipaddr", addr); return map; } }
dto
package com.et.ipfilter.dto; import lombok.AllArgsConstructor; import lombok.Data; import lombok.NoArgsConstructor; @Data @NoArgsConstructor @AllArgsConstructor public class CountryInfo { private String query; private String status; private String country; private String countryCode; private String region; private String regionName; private String city; private String zip; private String lat; private String lon; private String timezone; private String isp; private String org; private String as; }
Util
离线查询ip来源,
下载离线IP定位库
离线数据库在项目的data文件夹下,名称为ip2region.db
,其他2个文件是用于生成离线库的,可不用下载。
https://github.com/lionsoul2014/ip2region/tree/master/data/ip2region.xdb
下载到离线数据库后,我们需要读取这个数据库,我们可以放在项目的resources
目录
package com.et.ipfilter.util; import lombok.extern.slf4j.Slf4j; import org.lionsoul.ip2region.xdb.Searcher; import org.springframework.util.StringUtils; import javax.servlet.http.HttpServletRequest; /** * ip query */ @Slf4j public class IPOfflineUtil { private static final String UNKNOWN = "unknown"; protected IPOfflineUtil() { } public static String getIpAddr(HttpServletRequest request) { String ip = request.getHeader("X-Forwarded-For"); if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("Proxy-Client-IP"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("WL-Proxy-Client-IP"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("HTTP_X_FORWARDED_FOR"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("HTTP_X_FORWARDED"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("HTTP_X_CLUSTER_CLIENT_IP"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("HTTP_CLIENT_IP"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("HTTP_FORWARDED_FOR"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("HTTP_FORWARDED"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("HTTP_VIA"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getHeader("REMOTE_ADDR"); } if (!StringUtils.hasLength(ip) || UNKNOWN.equalsIgnoreCase(ip)) { ip = request.getRemoteAddr(); } int index = ip.indexOf(","); if (index != -1) { ip = ip.substring(0, index); } return "0:0:0:0:0:0:0:1".equals(ip) ? "127.0.0.1" : ip; } public static String getAddr(String ip) { String dbPath = "D:\\IdeaProjects\\ETFramework\\ipfilter\\src\\main\\resources\\ip2region\\ip2region.xdb"; // 1、from dbPath load all xdb to memory。 byte[] cBuff; try { cBuff = Searcher.loadContentFromFile(dbPath); } catch (Exception e) { log.info("failed to load content from `%s`: %s\n", dbPath, e); return null; } // 2、usr cBuff create a query object base on memory。 Searcher searcher; try { searcher = Searcher.newWithBuffer(cBuff); } catch (Exception e) { log.info("failed to create content cached searcher: %s\n", e); return null; } // 3、query try { String region = searcher.search(ip); return region; } catch (Exception e) { log.info("failed to search(%s): %s\n", ip, e); } return null; } }
在线查询访问ip
package com.et.ipfilter.util; import com.et.ipfilter.dto.CountryInfo; import org.springframework.http.ResponseEntity; import org.springframework.web.client.RestTemplate; import javax.annotation.PostConstruct; /** * @author liuhaihua * @version 1.0 * @ClassName IPOnlineUtil * @Description todo * @datphpe 2024/08/09/ 9:58 */ public class IPOnlineUtil { private static RestTemplate restTemplate; private final RestTemplate template; public static final String IP_API_URL = "http://ip-api.com/json/"; public IPOnlineUtil(RestTemplate restTemplate) { this.template = restTemplate; } /** * init RestTemplate */ @PostConstruct public void init() { setRestTemplate(this.template); } /** * init RestTemplate */ private static void setRestTemplate(RestTemplate template) { restTemplate = template; } /** * get country by ip * * @param ip * @return */ public static CountryInfo getCountryByIpOnline(String ip) { ResponseEntity<CountryInfo> entity = restTemplate.getForEntity( IP_API_URL + ip + "?lang=zh-CN", CountryInfo.class ); return entity.getBody(); } }
IP 查询工具类
package com.et.ipfilter.util; import org.lionsoul.ip2region.xdb.Searcher; import java.io.IOException; public class SearcherIPUtils { public static String getCachePosition(String ip) { return SearcherIPUtils.getCachePosition("src/main/resources/ip2region/ip2region.xdb", ip, true); } public static String getPosition(String dbPath,String ip,boolean format) { // 1、create searcher object Searcher searcher = null; try { searcher = Searcher.newWithFileOnly(dbPath); } catch (IOException e) { throw new RuntimeException(e); } // 2、query try { String region = searcher.search(ip); if (format){ return region; } String[] split = region.split("\\|"); String s = split[0] + split[2] + split[3]; return s; } catch (Exception e) { throw new RuntimeException(e); } } /** * @Description : * @Author : mabo */ public static String getIndexCachePosition(String dbPath, String ip, boolean format) { Searcher searcher = null; byte[] vIndex; try { vIndex = Searcher.loadVectorIndexFromFile(dbPath); } catch (Exception e) { throw new RuntimeException(e); } try { searcher = Searcher.newWithVectorIndex(dbPath, vIndex); } catch (Exception e) { throw new RuntimeException(e); } try { String region = searcher.search(ip); if (format){ return region; } String[] split = region.split("\\|"); String s = split[0] + split[2] + split[3]; return s; } catch (Exception e) { throw new RuntimeException(e); } } /** * @Description : * @Author : mabo */ public static String getCachePosition(String dbPath,String ip,boolean format) { byte[] cBuff; try { cBuff = Searcher.loadContentFromFile(dbPath); } catch (Exception e) { throw new RuntimeException(e); } Searcher searcher; try { searcher = Searcher.newWithBuffer(cBuff); } catch (Exception e) { throw new RuntimeException(e); } try { String region = searcher.search(ip); if (format){ return region; } String[] split = region.split("\\|"); String s = split[0] + split[2] + split[3]; return s; } catch (Exception e) { throw new RuntimeException(e); } } } package com.et.ipfilter.util; import com.alibaba.fastjson2.JSONObject; import lombok.extern.slf4j.Slf4j; import org.springframework.util.StringUtils; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.net.ConnectException; import java.net.SocketTimeoutException; import java.net.URL; import java.net.URLConnection; @Slf4j public class AddressUtils { public static final String IP_URL = "http://whois.pconline.com.cn/ipJson.JSP"; public static final String UNKNOWN = "XX XX"; public static String getRealAddressByIP(String ip) { String address = UNKNOWN; //if ip is inner ip,return if (internalIp(ip)) { return "inner IP"; } if (true) { try { String rspStr = sendGet(IP_URL, "ip=" + ip + "&json=true" ,"GBK"); if (StringUtils.isEmpty(rspStr)) { log.error("get addr exception {}" , ip); return UNKNOWN; } JSONObject obj = JSONObject.parseobject(rspStr); String region = obj.getString("pro"); String city = obj.getString("city"); return String.format("%s %s" , region, city); } catch (Exception e) { log.error("get addr exception {}" , ip); } } return address; } public static String sendGet(String url, String param, String contentType) { StringBuilder result = new StringBuilder(); BufferedReader in = null; try { String urlNameString = url + "?" + param; log.info("sendGet - {}" , urlNameString); URL realUrl = new URL(urlNameString); URLConnection connection = realUrl.openConnection(); connection.setRequestProperty("accept" , "*/*"); connection.setRequestProperty("connection" , "Keep-Alive"); connection.setRequestProperty("user-agent" , "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;SV1)"); connection.connect(); in = new BufferedReader(new InputStreamReader(connection.getInputStream(), contentType)); String line; while ((line = in.readLine()) != null) { result.append(line); } log.info("recv - {}" , result); } catch (ConnectException e) { log.error("invoke HttpUtils.sendGet ConnectException, url=" + url + ",param=" + param, e); } catch (SocketTimeoutException e) { log.error("invoke HttpUtils.sendGet SocketTimeoutException, url=" + url + ",param=" + param, e); } catch (IOException e) { log.error("invoke HttpUtils.sendGet IOException, url=" + url + ",param=" + param, e); } catch (Exception e) { log.error("invoke HttpsUtil.sendGet Exception, url=" + url + ",param=" + param, e); } finally { try { if (in != null) { in.close(); } } catch (Exception ex) { log.error("invoke in.close Exception, url=" + url + ",param=" + param, ex); } } return result.toString(); } /** * check the ip is inner? * * @param ip * @return */ public static boolean internalIp(String ip) { byte[] addr = textToNumericFormatV4php(ip); return internalIp(addr) || "127.0.0.1".equals(ip); } /** * check the ip is inner? * * @param addr * @return */ private static boolean internalIp(byte[] addr) { if (null==addr|| addr.length < 2) { return true; } final byte b0 = addr[0]; final byte b1 = addr[1]; // 10.x.x.x/8 final byte SECTION_1 = 0x0A; // 172.16.x.x/12 final byte SECTION_2 = (byte) 0xAC; final byte SECTION_3 = (byte) 0x10; final byte SECTION_4 = (byte) 0x1F; // 192.168.x.x/16 final byte SECTION_5 = (byte) 0xC0; final byte SECTION_6 = (byte) 0xA8; switch (b0) { case SECTION_1: return true; case SECTION_2: return (b1 >= SECTION_3 && b1 <= SECTION_4); case SECTION_5: return (b1 == SECTION_6); default: return false; } } /** * change IPv4 to byte * * @param text * @return byte */ private static byte[] textToNumericFormatV4(String text) { if (text.length() == 0) { return new byte[0]; } byte[] bytes = new byte[4]; String[] elements = text.split("\\.", -1); try { long l; int i; switch (elements.length) { case 1: l = Long.parseLong(elements[0]); if ((l < 0L) || (l > 4294967295L)) { return new byte[0]; } bytes[0] = (byte) (int) (l >> 24 & 0xFF); bytes[1] = (byte) (int) ((l & 0xFFFFFF) >> 16 & 0xFF); bytes[2] = (byte) (int) ((l & 0xFFFF) >> 8 & 0xFF); bytes[3] = (byte) (int) (l & 0xFF); break; case 2: l = Integer.parseInt(elements[0]); if ((l < 0L) || (l > 255L)) { return new byte[0]; } bytes[0] = (byte) (int) (l & 0xFF); l = Integer.parseInt(elements[1]); if ((l < 0L) || (l > 16777215L)) { return new byte[0]; } bytes[1] = (byte) (int) (l >> 16 & 0xFF); bytes[2] = (byte) (int) ((l & 0xFFFF) >> 8 & 0xFF); bytes[3] = (byte) (int) (l & 0xFF); break; case 3: for (i = 0; i < 2; ++i) { l = Integer.parseInt(elementsjavascript[i]); if ((l < 0L) || (l > 255L)) { return new byte[0]; } bytes[i] = (byte) (int) (l & 0xFF); } l = Integer.parseInt(elements[2]); if ((l < 0L) || (l > 65535L)) { return new byte[0]; } bytes[2] = (byte) (int) (l >> 8 & 0xFF); bytes[3] = (byte) (int) (l & 0xFF); break; case 4: for (i = 0; i < 4; ++i) { l = Integer.parseInt(elements[i]); if ((l < 0L) || (l > 255L)) { return new byte[0]; } bytes[i] = (byte) (int) (l & 0xFF); } break; default: return new byte[0]; } } catch (NumberFormatException e) { return new byte[0]; } return bytes; } }
以上只是一些关键代码,所有代码请参见下面代码仓库
代码仓库
- https://github.com/Harries/springboot-demo(ipfilter)
3.测试
- 启动 springBoot应用
- 访问http://127.0.0.1:8088/hello?ip=52.220.113.16
- 可以看到该ip属于新加坡,后续可以根据国家限制访问
以上就是SpringBoot集成ip2region实现ip白名单的代码示例的详细内容,更多关于SpringBoot ip白名单的资料请关注编程客栈(www.devze.com)其它相关文章!
精彩评论