Quentin/doluedu-fastapi-template
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor & reorder some points (#2)

Browse Source 
* Add clarifications for sqlalchemy, migrations, db points

* Add better explanation for background tasks

* Reorder points and break links
This commit is contained in:
Yerassyl
2022-08-26 01:27:51 +06:00
committed by GitHub
parent 9ec5226456
commit 308a7253e2
1 changed files with 57 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

95
README.md
View File

@@ -18,21 +18,19 @@ Some of them are worth sharing.
10. [Use Pydantic's BaseSettings for configs](https://github.com/zhanymkanov/fastapi-best-practices#10-use-pydantics-basesettings-for-configs) 10. [Use Pydantic's BaseSettings for configs](https://github.com/zhanymkanov/fastapi-best-practices#10-use-pydantics-basesettings-for-configs)
11. [SQLAlchemy: Set DB keys naming convention from day 0](https://github.com/zhanymkanov/fastapi-best-practices#11-sqlalchemy-set-db-keys-naming-convention-from-day-0) 11. [SQLAlchemy: Set DB keys naming convention from day 0](https://github.com/zhanymkanov/fastapi-best-practices#11-sqlalchemy-set-db-keys-naming-convention-from-day-0)
12. [Set DB table naming convention immediately from day 0](https://github.com/zhanymkanov/fastapi-best-practices#12-set-db-table-naming-convention-immediately-from-day-0) 12. [Set DB table naming convention immediately from day 0](https://github.com/zhanymkanov/fastapi-best-practices#12-set-db-table-naming-convention-immediately-from-day-0)
13. ~~Set UUIDs within the app~~ 13. [Set tests client async from day 0](https://github.com/zhanymkanov/fastapi-best-practices#14-set-tests-client-async-from-day-0)
14. [Set tests client async from day 0](https://github.com/zhanymkanov/fastapi-best-practices#14-set-tests-client-async-from-day-0) 14. [BackgroundTasks > asyncio.create_task](https://github.com/zhanymkanov/fastapi-best-practices#16-use-backgroundtasks)
15. ~~Set postgres identity from day 0~~ 15. [Typing is important](https://github.com/zhanymkanov/fastapi-best-practices#17-typing-is-important)
16. [Use BackgroundTasks](https://github.com/zhanymkanov/fastapi-best-practices#16-use-backgroundtasks) 16. [Save files in chunk](https://github.com/zhanymkanov/fastapi-best-practices#18-dont-hope-your-clients-will-send-small-blobs-save-files-in-chunk)
17. [Typing is important](https://github.com/zhanymkanov/fastapi-best-practices#17-typing-is-important) 17. [Be careful with dynamic pydantic fields](https://github.com/zhanymkanov/fastapi-best-practices#19-be-careful-with-dynamic-pydantic-fields)
18. [Don't hope your clients will send small BLOBs. Save files in chunk](https://github.com/zhanymkanov/fastapi-best-practices#18-dont-hope-your-clients-will-send-small-blobs-save-files-in-chunk) 18. ~~SQL-first, Pydantic-second, Custom-third~~
19. [Be careful with dynamic pydantic fields](https://github.com/zhanymkanov/fastapi-best-practices#19-be-careful-with-dynamic-pydantic-fields) 19. [Validate url source (if users are able to upload files)](https://github.com/zhanymkanov/fastapi-best-practices#22-validate-url-source-if-users-are-able-to-upload-files)
20. ~~SQL-first, Pydantic-second~~ 20. [root_validator to use multiple columns during validation](https://github.com/zhanymkanov/fastapi-best-practices#23-root_validator-to-use-multiple-columns-during-validation)
21. ~~Validate file formats~~ 21. ~~pre=True if data need to be pre-handled before validation~~
22. [Validate url source (if users are able to upload files)](https://github.com/zhanymkanov/fastapi-best-practices#22-validate-url-source-if-users-are-able-to-upload-files) 22. ~~raise a ValueError in pydantic, if schema faces http client~~
23. [root_validator to use multiple columns during validation](https://github.com/zhanymkanov/fastapi-best-practices#23-root_validator-to-use-multiple-columns-during-validation) 23. ~~remember fastapi response modeling~~
24. ~~pre=True if data need to be pre-handled before validation~~ 24. if must use sdk, but it's not async, use threadpools
25. ~~raise a ValueError in pydantic, if schema faces http client~~ 25. ~~use linters~~
26. ~~remember fastapi response modeling~~
27. if must use sdk, but it's not async, use threadpools
### 1. Project Structure. Consistent & predictable ### 1. Project Structure. Consistent & predictable
There are many ways to structure the project, but the best structure is a structure that is consistent, straightforward and has no surprises. There are many ways to structure the project, but the best structure is a structure that is consistent, straightforward and has no surprises.
@@ -42,7 +40,7 @@ There are many ways to structure the project, but the best structure is a struct
- If the frequency and location of the files feels random, then your project structure is bad. - If the frequency and location of the files feels random, then your project structure is bad.
Although, the project structure, where we separate files by their type (e.g. api, crud, models, schemas) Although, the project structure, where we separate files by their type (e.g. api, crud, models, schemas)
presented by [@tiangolo](https://github.com/tiangolo) is perfect for microservices or projects with fewer scopes, presented by [@tiangolo](https://github.com/tiangolo) is good for microservices or projects with fewer scopes,
we couldn't fit it into our monolith with a lot of domains and modules. we couldn't fit it into our monolith with a lot of domains and modules.
Structure that I found more scalable and evolvable is inspired by Netflix's [Dispatch](https://github.com/Netflix/dispatch) with some little modifications. Structure that I found more scalable and evolvable is inspired by Netflix's [Dispatch](https://github.com/Netflix/dispatch) with some little modifications.
``` ```
@@ -336,7 +334,7 @@ Use /me endpoints for users own resources (e.g. `GET /profiles/me`, `GET /users/
### 7. Don't make your routes async, if you have only blocking I/O operations ### 7. Don't make your routes async, if you have only blocking I/O operations
Under the hood, FastAPI can [effectively handle](https://fastapi.tiangolo.com/async/#path-operation-functions) both async and sync I/O operations. Under the hood, FastAPI can [effectively handle](https://fastapi.tiangolo.com/async/#path-operation-functions) both async and sync I/O operations.
- FastAPI calls sync routes in the [threadpool](https://en.wikipedia.org/wiki/Thread_pool) - FastAPI runs `sync` routes in the [threadpool](https://en.wikipedia.org/wiki/Thread_pool)
and blocking I/O operations won't stop [event loop](https://docs.python.org/3/library/asyncio-eventloop.html) and blocking I/O operations won't stop [event loop](https://docs.python.org/3/library/asyncio-eventloop.html)
from executing the tasks. from executing the tasks.
- Otherwise, if the route is defined as `async` then it's called regularly via `await` - Otherwise, if the route is defined as `async` then it's called regularly via `await`
@@ -528,7 +526,8 @@ class AppSettings(BaseSettings):
IS_GOOD_ENV: bool = True IS_GOOD_ENV: bool = True
ALLOWED_CORS_ORIGINS: set[AnyUrl] ALLOWED_CORS_ORIGINS: set[AnyUrl]
``` ```
### 11. SQLAlchemy: Set DB keys naming convention from day 0 ### 11. SQLAlchemy: Set DB keys naming convention
Explicitly setting the indexes' namings according to your database's convention is preferable over sqlalchemy's.
```python ```python
from sqlalchemy import MetaData from sqlalchemy import MetaData
@@ -541,12 +540,31 @@ POSTGRES_INDEXES_NAMING_CONVENTION = {
} }
metadata = MetaData(naming_convention=POSTGRES_INDEXES_NAMING_CONVENTION) metadata = MetaData(naming_convention=POSTGRES_INDEXES_NAMING_CONVENTION)
``` ```
### 12. Set DB table naming convention immediately from day 0 ### 12. Migrations. Alembic.
### 13. Set UUIDs within the app 1. Migrations must be static and easily revertable.
Setting them in database makes it harder to write integration tests. If your migrations depend on dynamically generated data,
make sure the only thing that changes there is the data itself, not its structure.
2. Generate migrations with descriptive name & slug. Slug is required and should explain the changes.
Set human-readable file template for new migrations.
We use `date` + `slug` pattern, e.g. `2022-08-24_post_content_idx.py`
```
# alembic.ini
file_template = %%(year)d-%%(month).2d-%%(day).2d_%%(slug)s
```
### 13. Set DB tables naming convention
Being consistent with names was important. Some of the rules we followed:
1. lower_case_snake
2. singular form
3. group similar tables with module prefix, e.g. `payment_account`, `payment_bill`, `post`, `post_like`
4. stay consistent across tables, but concrete namings are ok, e.g.
1. use `profile_id` in all tables, but if some of them need only profiles that are creators, use `creator_id`
2. use `post_id` for all abstract tables like `post_like`, `post_view`, but use concrete naming in relevant modules like `course_id` in `chapters.course_id`
5. `_at` suffix for datetime
6. `_date` suffix for date
### 14. Set tests client async from day 0 ### 14. Set tests client async from day 0
Writing integration tests with DB will most likely lead to messed up event loop errors in the future. Writing integration tests with DB will most likely lead to messed up event loop errors in the future.
Set the async test client immediately, e.g. [asyn_asgi_testclient](https://github.com/vinissimus/async-asgi-testclient) or [httpx](https://github.com/encode/starlette/issues/652) Set the async test client immediately, e.g. [async_asgi_testclient](https://github.com/vinissimus/async-asgi-testclient) or [httpx](https://github.com/encode/starlette/issues/652)
```python ```python
import pytest import pytest
from async_asgi_testclient import TestClient from async_asgi_testclient import TestClient
@@ -572,9 +590,11 @@ async def test_create_post(client: TestClient):
assert resp.status_code == 201 assert resp.status_code == 201
``` ```
Unless you have sync db connection (excuse me?) or aren't planning to write integration tests. Unless you have sync db connection (excuse me?) or aren't planning to write integration tests.
### 15. Set postgres identity from day 0 ### 15. BackgroundTasks > asyncio.create_task
### 16. Use BackgroundTasks BackgroundTasks can [effectively run](https://github.com/encode/starlette/blob/31164e346b9bd1ce17d968e1301c3bb2c23bb418/starlette/background.py#L25) both blocking and non-blocking I/O operations.
They are stable enough for async (delayed) tasks Since the API for sending these tasks will be the same (no need to await coroutines),
it's preferable to use starlette's background tasks.
Don't use it for CPU intensive tasks.
```python ```python
from fastapi import BackgroundTasks from fastapi import BackgroundTasks
from pydantic import UUID4 from pydantic import UUID4
@@ -589,7 +609,7 @@ async def send_user_email(worker: BackgroundTasks, user_id: UUID4):
worker.add_task(notifications_service.send_email, user_id) # send email after responding client worker.add_task(notifications_service.send_email, user_id) # send email after responding client
return {"status": "ok"} return {"status": "ok"}
``` ```
### 17. Typing is important ### 16. Typing is important
FastAPI, Pydantic, and modern IDEs encourage to take use of type hints. FastAPI, Pydantic, and modern IDEs encourage to take use of type hints.
**Without Type Hints** **Without Type Hints**
@@ -600,7 +620,8 @@ FastAPI, Pydantic, and modern IDEs encourage to take use of type hints.
<img src="images/type_hints.png" width="400" height="auto"> <img src="images/type_hints.png" width="400" height="auto">
### 18. Don't hope your clients will send small BLOBs. Save files in chunk. ### 17. Save files in chunk.
Don't hope your clients will send small files.
```python ```python
import aiofiles import aiofiles
from fastapi import UploadFile from fastapi import UploadFile
@@ -612,7 +633,7 @@ async def save_video(video_file: UploadFile):
while chunk := await video_file.read(DEFAULT_CHUNK_SIZE): while chunk := await video_file.read(DEFAULT_CHUNK_SIZE):
await f.write(chunk) await f.write(chunk)
``` ```
### 19. Be careful with dynamic pydantic fields ### 18. Be careful with dynamic pydantic fields
If you have a pydantic field that can accept multiple types, be sure validator explicitly knows the difference between those types. If you have a pydantic field that can accept multiple types, be sure validator explicitly knows the difference between those types.
```python ```python
from pydantic import BaseModel from pydantic import BaseModel
@@ -719,9 +740,8 @@ print(type(p.field_2))
print(type(p.content)) print(type(p.content))
# OUTPUT: Article # OUTPUT: Article
``` ```
### 20. SQL-first, Pydantic-second ### 19. SQL-first, Pydantic-second, Custom-last
### 21. Validate file formats ### 20. Validate url source (if users are able to upload files and send urls)
### 22. Validate url source (if users are able to upload files)
Bad users could send strange urls for user facing public objects. Bad users could send strange urls for user facing public objects.
```python ```python
from pydantic import AnyUrl, BaseModel from pydantic import AnyUrl, BaseModel
@@ -744,7 +764,7 @@ class Post(BaseModel):
thumbnail_url: CompanyMediaUrl thumbnail_url: CompanyMediaUrl
``` ```
### 23. root_validator to use multiple columns during validation ### 21. root_validator to use multiple columns during validation
```python ```python
from pydantic import BaseModel, root_validator from pydantic import BaseModel, root_validator
@@ -761,11 +781,10 @@ class Profile(BaseModel):
return data return data
``` ```
### 24. pre if data need to be pre-handled before validation ### 22. pre if data need to be pre-handled before validation
### 25. you can just raise a ValueError in pydantic schemas, if schemas faces http client ### 23. you can just raise a ValueError in pydantic schemas, if schemas faces http client
it wil return a nice response it wil return a nice response
### 26. don't forget that fastapi converts response Model to Dict then to Model then to JSON ### 24. don't forget that fastapi converts response Model to Dict then to Model then to JSON
it may lead to bugs like model can parse only raw data (e.g. forced data aggregation for raw data) it may lead to bugs like model can parse only raw data (e.g. forced data aggregation for raw data)
### 27. if must use sdk, but it's not async, use threadpools ### 25. if must use sdk, but it's not async, use threadpools
### 28. use linters (black, isort, autoflake) ### 26. use linters (black, isort, autoflake)
### 29. set logs from day 0